08/04/2025 02:13:41.084 7f032b138f48 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 08/04/2025 02:13:41.084 7f032b138f48 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 08/04/2025 02:13:41.084 7f032b138f48 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 08/04/2025 02:13:41.094 7f03286298d0 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port 08/04/2025 02:13:41.094 7f03286298d0 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112 08/04/2025 02:13:41.095 7f0328630240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 08/04/2025 02:13:41.095 7f0328630240 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 08/04/2025 02:13:41.096 7f0328630240 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 08/04/2025 02:13:41.096 7f0328630240 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 08/04/2025 02:13:41.096 7f0328630240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:41.096 7f0328630240 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 08/04/2025 02:13:41.096 7f0328630240 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:41.097 7f0328630240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:41.097 7f0328630240 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 08/04/2025 02:13:41.097 7f0328630240 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:41.275 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 99457835857142976279207433248171955532985935287919770684990740624479076768054, 'start': '1754273618000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 92966901289344716829206930100099110015716407404192304273242993660857206963722, 'start': '1754273618000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 11159768657091199521267261969118247599164202887999258888163851382557156371291, 'start': '1754273619000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 08/04/2025 02:13:41.433 7f03312d9a20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:41.434 7f03312d9a20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-08-04 00:44:07.888545", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10117-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 8 num_core_per_socket = 8 num_thread_per_core = 1 >> running_package_version = nightly-nightly.el7 installed_package_version = nightly-nightly.el7 >> 08/04/2025 02:13:41.434 7f03312d9a20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:41.435 7f032b138f48 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3749-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1754273621435189, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3749-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 08/04/2025 02:13:41.435 7f032b138f48 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3749-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1754273621435658, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3749-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1754273618000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1754273618000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1754273619000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-08-04 00:44:07.888545", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10117-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 08/04/2025 02:13:41.437 7f032864e2e8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:41.437 7f032864e2e8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 08/04/2025 02:13:41.437 7f032864e2e8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:41.437 7f032864e2e8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:41.437 7f032864e2e8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1754273618000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1754273618000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1754273619000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-08-04 00:44:07.888545", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10117-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 08/04/2025 02:13:41.437 7f032864e2e8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:41.779 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 08/04/2025 02:13:42.164 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 08/04/2025 02:13:42.165 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:42.165 7f032ac444e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. database_usage = [ << disk_space_used_1k = 13609020 disk_space_available_1k = 67481464 config_db_size_1k = 108 >>, ] >> 08/04/2025 02:13:42.165 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:42.595 7f032b138f48 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL and socket: Result: 1 08/04/2025 02:13:42.597 7f032b138f48 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3749-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1754273622597178, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3749-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 08/04/2025 02:13:42.598 7f03274cb048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:42.598 7f03274cb048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 08/04/2025 02:13:42.598 7f03274cb048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:43.780 7f032b138f48 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/04/2025 02:13:45.066 7f032b138f48 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1 08/04/2025 02:13:45.095 7f03274cb588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:13:45.101 7f03274cb588 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:13:45.101 7f03274cb588 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:13:45.101 7f03274cb588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:13:45.101 7f03274cb588 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:13:45.101 7f03274cb588 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:13:45.101 7f03274cb588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:13:45.102 7f03274cb588 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:13:45.103 7f03274cb588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:45.103 7f03274cb588 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:13:45.103 7f03274cb588 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:45.103 7f03274cb588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:45.104 7f03274cb588 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:13:45.104 7f03274cb588 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:45.988 7f032b138f48 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1 08/04/2025 02:13:46.927 7f032b138f48 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1 08/04/2025 02:13:47.837 7f032b138f48 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1 08/04/2025 02:13:48.779 7f032b138f48 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1 08/04/2025 02:13:49.102 7f0328658eb8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:13:49.103 7f0328658eb8 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:13:49.103 7f0328658eb8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:13:49.103 7f0328658eb8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:13:49.103 7f0328658eb8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:13:49.103 7f0328658eb8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:13:49.103 7f0328658eb8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:13:49.104 7f0328658eb8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:13:49.104 7f0328658eb8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:49.104 7f0328658eb8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:13:49.104 7f0328658eb8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:49.104 7f0328658eb8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:49.104 7f0328658eb8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:13:49.104 7f0328658eb8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:49.669 7f032b138f48 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1 08/04/2025 02:13:53.104 7f03273b2390 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:13:53.156 7f03273b2390 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:13:53.157 7f03273b2390 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:13:53.157 7f03273b2390 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:13:53.157 7f03273b2390 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:13:53.157 7f03273b2390 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:13:53.161 7f03273b2390 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:13:53.163 7f03273b2390 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:13:53.163 7f03273b2390 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:53.163 7f03273b2390 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:13:53.163 7f03273b2390 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:53.164 7f03273b2390 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:53.164 7f03273b2390 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:13:53.164 7f03273b2390 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:54.698 7f032b138f48 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3749-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1754273634698688, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3749-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1754273618000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1754273618000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1754273619000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-08-04 00:44:07.888545", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10117-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 08/04/2025 02:13:54.699 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3749-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13852588, partition_space_available_1k=67237896, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1218476, cpu_share=0.0, mem_res=1205960, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50400, cpu_share=0.0, mem_res=48024, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52320, cpu_share=0.0, mem_res=40076, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=5132736, free=13185756, buffers=612820, cached=13733800, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.57, five_min_avg=1.94, fifteen_min_avg=1.2, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 08/04/2025 02:13:54.699 7f03274cbcc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:54.699 7f03274cbcc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1754273618000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1754273618000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1754273619000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-08-04 00:44:07.888545", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10117-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 08/04/2025 02:13:54.699 7f03274cbcc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:54.699 7f03274cbcc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:54.700 7f03274cbcc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 13852588 partition_space_available_1k = 67237896 percentage_partition_space_used = 17 >> } all_core_file_list = [ ] process_mem_cpu_usage = { cassandra : << mem_virt = 1218476 cpu_share = 0.0 mem_res = 1205960 >>zookeeper : << mem_virt = 50400 cpu_share = 0.0 mem_res = 48024 >>contrail-config-database-nodemgr : << mem_virt = 52320 cpu_share = 0.0 mem_res = 40076 >> } system_mem_usage = << total = 32665112 used = 5132736 free = 13185756 buffers = 612820 cached = 13733800 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 3.57 five_min_avg = 1.94 fifteen_min_avg = 1.2 cpu_share = 0 node_type = config-database-node >> >> 08/04/2025 02:13:54.700 7f03274cbcc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:57.157 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:13:57.158 7f0330d9d240 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:13:57.158 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:13:57.158 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:13:57.159 7f0330d9d240 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:13:57.159 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:13:57.159 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:13:57.159 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:13:57.159 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:57.159 7f0330d9d240 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:13:57.159 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:57.160 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:13:57.160 7f0330d9d240 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:13:57.160 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:01.162 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:14:01.164 7f0330d9d240 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:14:01.164 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:14:01.164 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:01.165 7f0330d9d240 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:14:01.165 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:01.165 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:14:01.165 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:14:01.165 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:01.165 7f0330d9d240 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:14:01.166 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:01.166 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:01.166 7f0330d9d240 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:14:01.166 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:05.166 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:14:05.168 7f0330d9d240 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:14:05.168 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:14:05.168 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:05.168 7f0330d9d240 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:14:05.168 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:05.168 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:14:05.169 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:14:05.169 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:05.169 7f0330d9d240 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:14:05.169 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:05.169 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:05.169 7f0330d9d240 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:14:05.169 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:09.171 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:14:09.173 7f0330d9d240 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:14:09.173 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:14:09.173 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:09.173 7f0330d9d240 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:14:09.173 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:09.174 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:14:09.174 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:14:09.174 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:09.174 7f0330d9d240 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:14:09.175 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:09.175 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:09.175 7f0330d9d240 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:14:09.176 7f0330d9d240 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:13.174 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:14:13.176 7f032865c630 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:14:13.176 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:14:13.176 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:13.177 7f032865c630 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:14:13.177 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:13.177 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:14:13.178 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:14:13.178 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:13.178 7f032865c630 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:14:13.178 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:13.178 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:13.178 7f032865c630 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:14:13.178 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:17.180 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:14:17.181 7f032865c630 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:14:17.181 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:14:17.181 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:17.181 7f032865c630 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:14:17.181 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:17.181 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:14:17.182 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:14:17.182 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:17.182 7f032865c630 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:14:17.182 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:17.182 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:17.182 7f032865c630 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:14:17.182 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:21.186 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:14:21.187 7f032865c630 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:14:21.187 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:14:21.187 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:21.187 7f032865c630 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:14:21.187 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:21.187 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:14:21.188 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:14:21.188 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:21.188 7f032865c630 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:14:21.188 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:21.188 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:21.188 7f032865c630 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:14:21.188 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:25.191 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:14:25.192 7f032865c630 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:14:25.193 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:14:25.193 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:25.193 7f032865c630 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:14:25.193 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:25.193 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:14:25.193 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:14:25.194 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:25.194 7f032865c630 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:14:25.194 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:25.194 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:25.194 7f032865c630 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:14:25.194 7f032865c630 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:29.194 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:14:29.195 7f032aeb0a20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:14:29.196 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:14:29.196 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:29.196 7f032aeb0a20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:14:29.196 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:29.196 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:14:29.197 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:14:29.197 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:29.197 7f032aeb0a20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:14:29.198 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:29.198 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:29.198 7f032aeb0a20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:14:29.198 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:33.198 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:14:33.199 7f032aeb0a20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:14:33.200 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:14:33.200 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:33.200 7f032aeb0a20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:14:33.200 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:33.200 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:14:33.200 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:14:33.200 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:33.200 7f032aeb0a20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:14:33.201 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:33.201 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:33.201 7f032aeb0a20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:14:33.201 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:37.202 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:14:37.203 7f032aeb0a20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:14:37.203 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:14:37.204 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:37.204 7f032aeb0a20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:14:37.204 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:37.204 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:14:37.204 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:14:37.205 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:37.205 7f032aeb0a20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:14:37.205 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:37.205 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:37.205 7f032aeb0a20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:14:37.205 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:41.206 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:14:41.208 7f032aeb0a20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:14:41.208 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:14:41.209 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:41.209 7f032aeb0a20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:14:41.210 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:41.210 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:14:41.210 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:14:41.211 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:41.211 7f032aeb0a20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:14:41.211 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:41.211 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:41.211 7f032aeb0a20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:14:41.211 7f032aeb0a20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:41.760 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 08/04/2025 02:14:43.515 7f03277dc2e8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:43.516 7f03277dc2e8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. database_usage = [ << disk_space_used_1k = 14583400 disk_space_available_1k = 66507084 config_db_size_1k = 1648 >>, ] >> 08/04/2025 02:14:43.516 7f03277dc2e8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:44.153 7f032b138f48 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3749-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1754273684153625, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3749-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.249:8086'], status='Down', description='Connect to Idle on EvTcpConnectFail')]), flag_infos=None, description='Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 08/04/2025 02:14:44.156 7f032af38588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:44.156 7f032af38588 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:14:44.156 7f032af38588 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:45.144 7f032b138f48 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/04/2025 02:14:45.208 7f0327814438 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:14:45.210 7f0327814438 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:14:45.210 7f0327814438 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:14:45.211 7f0327814438 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:45.211 7f0327814438 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:14:45.211 7f0327814438 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:45.211 7f0327814438 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:14:45.212 7f0327814438 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:14:45.212 7f0327814438 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:45.212 7f0327814438 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:14:45.212 7f0327814438 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:45.213 7f0327814438 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:45.213 7f0327814438 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:14:45.213 7f0327814438 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:49.211 7f03273c4240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:14:49.212 7f03273c4240 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:14:49.212 7f03273c4240 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:14:49.213 7f03273c4240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:49.213 7f03273c4240 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:14:49.213 7f03273c4240 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:49.213 7f03273c4240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:14:49.213 7f03273c4240 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:14:49.213 7f03273c4240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:49.214 7f03273c4240 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:14:49.214 7f03273c4240 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:49.214 7f03273c4240 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:49.214 7f03273c4240 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:14:49.214 7f03273c4240 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:50.188 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3749-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14583840, partition_space_available_1k=66506644, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1917896, cpu_share=0.74, mem_res=1890440, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=76320, cpu_share=0.02, mem_res=73244, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52712, cpu_share=0.02, mem_res=40360, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=6353596, free=11301172, buffers=663724, cached=14346620, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.64, five_min_avg=2.2, fifteen_min_avg=1.33, cpu_share=4.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 08/04/2025 02:14:50.188 7f032862d390 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:50.189 7f032862d390 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 14583840 partition_space_available_1k = 66506644 percentage_partition_space_used = 18 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1917896 cpu_share = 0.74 mem_res = 1890440 >>zookeeper : << mem_virt = 76320 cpu_share = 0.02 mem_res = 73244 >>contrail-config-database-nodemgr : << mem_virt = 52712 cpu_share = 0.02 mem_res = 40360 >> } system_mem_usage = << total = 32665112 used = 6353596 free = 11301172 buffers = 663724 cached = 14346620 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 3.64 five_min_avg = 2.2 fifteen_min_avg = 1.33 cpu_share = 4.5 node_type = config-database-node >> >> 08/04/2025 02:14:50.189 7f032862d390 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:53.216 7f032862d390 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:14:53.217 7f032862d390 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:14:53.218 7f032862d390 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:14:53.218 7f032862d390 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:53.218 7f032862d390 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:14:53.218 7f032862d390 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:53.218 7f032862d390 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:14:53.218 7f032862d390 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:14:53.219 7f032862d390 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:53.219 7f032862d390 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:14:53.219 7f032862d390 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:53.219 7f032862d390 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:53.219 7f032862d390 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:14:53.219 7f032862d390 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:57.222 7f032862d390 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:14:57.223 7f032862d390 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:14:57.224 7f032862d390 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:14:57.224 7f032862d390 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:57.224 7f032862d390 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:14:57.224 7f032862d390 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:14:57.224 7f032862d390 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:14:57.224 7f032862d390 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:14:57.224 7f032862d390 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:57.225 7f032862d390 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:14:57.225 7f032862d390 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:57.225 7f032862d390 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:14:57.225 7f032862d390 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:14:57.225 7f032862d390 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:01.226 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:15:01.227 7f032c348c18 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:15:01.227 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:15:01.227 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:01.228 7f032c348c18 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:15:01.228 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:01.228 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:15:01.228 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:15:01.228 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:01.228 7f032c348c18 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:15:01.228 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:01.228 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:01.229 7f032c348c18 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:15:01.229 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:05.232 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:15:05.234 7f032c348c18 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:15:05.234 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:15:05.234 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:05.234 7f032c348c18 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:15:05.234 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:05.235 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:15:05.235 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:15:05.235 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:05.235 7f032c348c18 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:15:05.235 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:05.235 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:05.235 7f032c348c18 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:15:05.236 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:09.234 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:15:09.236 7f032c348c18 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:15:09.236 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:15:09.236 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:09.236 7f032c348c18 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:15:09.237 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:09.237 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:15:09.237 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:15:09.238 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:09.238 7f032c348c18 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:15:09.238 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:09.238 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:09.239 7f032c348c18 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:15:09.239 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:13.240 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:15:13.242 7f032c348c18 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:15:13.242 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:15:13.242 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:13.242 7f032c348c18 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:15:13.242 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:13.242 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:15:13.242 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:15:13.243 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:13.243 7f032c348c18 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:15:13.243 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:13.243 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:13.243 7f032c348c18 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:15:13.243 7f032c348c18 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:17.245 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:15:17.247 7f03286c1f60 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:15:17.247 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:15:17.247 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:17.247 7f03286c1f60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:15:17.247 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:17.247 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:15:17.248 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:15:17.248 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:17.248 7f03286c1f60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:15:17.248 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:17.248 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:17.248 7f03286c1f60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:15:17.248 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:21.250 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:15:21.252 7f03286c1f60 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:15:21.252 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:15:21.252 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:21.252 7f03286c1f60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:15:21.252 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:21.252 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:15:21.253 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:15:21.253 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:21.253 7f03286c1f60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:15:21.253 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:21.253 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:21.253 7f03286c1f60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:15:21.253 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:25.254 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:15:25.255 7f03286c1f60 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:15:25.255 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:15:25.255 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:25.256 7f03286c1f60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:15:25.256 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:25.256 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:15:25.256 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:15:25.256 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:25.256 7f03286c1f60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:15:25.256 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:25.257 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:25.257 7f03286c1f60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:15:25.257 7f03286c1f60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:29.256 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:15:29.258 7f032866b4e0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:15:29.258 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:15:29.258 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:29.258 7f032866b4e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:15:29.258 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:29.258 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:15:29.259 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:15:29.259 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:29.259 7f032866b4e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:15:29.259 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:29.259 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:29.259 7f032866b4e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:15:29.259 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:33.262 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:15:33.263 7f032866b4e0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:15:33.264 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:15:33.264 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:33.264 7f032866b4e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:15:33.264 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:33.264 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:15:33.264 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:15:33.264 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:33.265 7f032866b4e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:15:33.265 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:33.265 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:33.265 7f032866b4e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:15:33.265 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:37.267 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:15:37.268 7f032866b4e0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:15:37.268 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:15:37.268 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:37.268 7f032866b4e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:15:37.269 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:37.269 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:15:37.269 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:15:37.269 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:37.269 7f032866b4e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:15:37.269 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:37.269 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:37.270 7f032866b4e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:15:37.270 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:41.270 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:15:41.271 7f032866b4e0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:15:41.271 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:15:41.271 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:41.272 7f032866b4e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:15:41.272 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:41.272 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:15:41.272 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:15:41.272 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:41.272 7f032866b4e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:15:41.273 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:41.273 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:41.273 7f032866b4e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:15:41.273 7f032866b4e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:41.769 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 08/04/2025 02:15:42.163 7f03273c4198 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:42.163 7f03273c4198 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. database_usage = [ << disk_space_used_1k = 15419636 disk_space_available_1k = 65670848 config_db_size_1k = 1648 >>, ] >> 08/04/2025 02:15:42.163 7f03273c4198 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:43.815 7f032b138f48 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/04/2025 02:15:45.272 7f03273b2ac8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:15:45.274 7f03273b2ac8 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:15:45.274 7f03273b2ac8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:15:45.275 7f03273b2ac8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:45.275 7f03273b2ac8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:15:45.275 7f03273b2ac8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:45.275 7f03273b2ac8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:15:45.275 7f03273b2ac8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:15:45.276 7f03273b2ac8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:45.276 7f03273b2ac8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:15:45.276 7f03273b2ac8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:45.276 7f03273b2ac8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:45.276 7f03273b2ac8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:15:45.276 7f03273b2ac8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:48.853 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3749-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15419788, partition_space_available_1k=65670696, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1931860, cpu_share=0.27, mem_res=1904096, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=80564, cpu_share=0.01, mem_res=77452, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52792, cpu_share=0.02, mem_res=40360, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=6804376, free=9910616, buffers=687356, cached=15262764, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.99, five_min_avg=2.23, fifteen_min_avg=1.4, cpu_share=3.38, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 08/04/2025 02:15:48.853 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:48.854 7f032865c780 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 15419788 partition_space_available_1k = 65670696 percentage_partition_space_used = 19 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1931860 cpu_share = 0.27 mem_res = 1904096 >>zookeeper : << mem_virt = 80564 cpu_share = 0.01 mem_res = 77452 >>contrail-config-database-nodemgr : << mem_virt = 52792 cpu_share = 0.02 mem_res = 40360 >> } system_mem_usage = << total = 32665112 used = 6804376 free = 9910616 buffers = 687356 cached = 15262764 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 2.99 five_min_avg = 2.23 fifteen_min_avg = 1.4 cpu_share = 3.38 node_type = config-database-node >> >> 08/04/2025 02:15:48.854 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:49.275 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:15:49.277 7f032865c780 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:15:49.277 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:15:49.277 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:49.277 7f032865c780 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:15:49.277 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:49.277 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:15:49.278 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:15:49.278 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:49.278 7f032865c780 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:15:49.278 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:49.278 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:49.279 7f032865c780 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:15:49.279 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:53.282 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:15:53.283 7f032865c780 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:15:53.283 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:15:53.284 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:53.284 7f032865c780 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:15:53.284 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:53.284 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:15:53.284 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:15:53.284 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:53.285 7f032865c780 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:15:53.285 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:53.285 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:53.285 7f032865c780 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:15:53.285 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:57.286 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:15:57.287 7f032865c780 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:15:57.287 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:15:57.288 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:57.288 7f032865c780 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:15:57.288 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:15:57.288 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:15:57.288 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:15:57.289 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:57.289 7f032865c780 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:15:57.289 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:57.289 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:15:57.289 7f032865c780 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:15:57.289 7f032865c780 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:01.290 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:16:01.292 7f0328630588 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:16:01.292 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:16:01.292 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:16:01.292 7f0328630588 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:16:01.292 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:16:01.292 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:16:01.293 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:16:01.293 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:01.293 7f0328630588 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:16:01.294 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:01.294 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:01.294 7f0328630588 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:16:01.294 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:05.296 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:16:05.298 7f0328630588 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:16:05.298 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:16:05.298 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:16:05.299 7f0328630588 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:16:05.299 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:16:05.299 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:16:05.300 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:16:05.302 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:05.302 7f0328630588 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:16:05.302 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:05.303 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:05.303 7f0328630588 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:16:05.303 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:09.302 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:16:09.303 7f0328630588 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:16:09.303 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:16:09.304 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:16:09.304 7f0328630588 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:16:09.304 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:16:09.304 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:16:09.304 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:16:09.304 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:09.304 7f0328630588 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:16:09.305 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:09.305 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:09.305 7f0328630588 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:16:09.305 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:13.306 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:16:13.307 7f0328630588 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:16:13.308 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:16:13.308 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:16:13.308 7f0328630588 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:16:13.308 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:16:13.308 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:16:13.308 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:16:13.308 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:13.309 7f0328630588 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:16:13.309 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:13.309 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:13.309 7f0328630588 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:16:13.309 7f0328630588 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:17.310 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:16:17.311 7f032ac444e0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:16:17.311 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:16:17.311 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:16:17.312 7f032ac444e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:16:17.312 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:16:17.312 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:16:17.312 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:16:17.312 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:17.313 7f032ac444e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:16:17.313 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:17.313 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:17.313 7f032ac444e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:16:17.313 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:21.316 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:16:21.317 7f032ac444e0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:16:21.317 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:16:21.317 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:16:21.317 7f032ac444e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:16:21.317 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:16:21.318 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:16:21.318 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:16:21.318 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:21.318 7f032ac444e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:16:21.318 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:21.319 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:21.319 7f032ac444e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:16:21.319 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:25.318 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:16:25.320 7f032ac444e0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:16:25.320 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:16:25.320 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:16:25.320 7f032ac444e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:16:25.320 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:16:25.320 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:16:25.321 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:16:25.321 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:25.321 7f032ac444e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:16:25.321 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:25.321 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:25.322 7f032ac444e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:16:25.322 7f032ac444e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:29.320 7f03274cb048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:16:29.321 7f03274cb048 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:16:29.322 7f03274cb048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:16:29.322 7f03274cb048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:16:29.322 7f03274cb048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:16:29.322 7f03274cb048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:16:29.322 7f03274cb048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:16:29.322 7f03274cb048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:16:29.322 7f03274cb048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:29.323 7f03274cb048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:16:29.323 7f03274cb048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:29.323 7f03274cb048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:29.323 7f03274cb048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:16:29.323 7f03274cb048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:33.325 7f03273d0ac8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:16:33.327 7f03273d0ac8 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 08/04/2025 02:16:33.327 7f03273d0ac8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:16:33.327 7f03273d0ac8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:16:33.327 7f03273d0ac8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:16:33.327 7f03273d0ac8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:16:33.327 7f03273d0ac8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/04/2025 02:16:33.328 7f03273d0ac8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/04/2025 02:16:33.328 7f03273d0ac8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:33.328 7f03273d0ac8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/04/2025 02:16:33.328 7f03273d0ac8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:33.328 7f03273d0ac8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:33.328 7f03273d0ac8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1754273621096218 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/04/2025 02:16:33.329 7f03273d0ac8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/04/2025 02:16:37.330 7f03273d0ac8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/04/2025 02:16:37.331 7f03273d0ac8 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 08/04/2025 02:16:37.332 7f03273d0ac8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/04/2025 02:16:37.332 7f03273d0ac8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:16:37.332 7f03273d0ac8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3749-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/04/2025 02:16:37.332 7f03273d0ac8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/04/2025 02:16:37.332 7f03273d0ac8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 08/04/2025 02:16:37.333 7f03273d0ac8 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 08/04/2025 02:16:37.334 7f03273d0ac8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 08/04/2025 02:16:37.334 7f03273d0ac8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 08/04/2025 02:16:37.334 7f03273d0ac8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 08/04/2025 02:16:41.775 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 08/04/2025 02:16:42.272 7f0328670438 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 08/04/2025 02:16:43.887 7f032af1ea20 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016 08/04/2025 02:16:43.888 7f032af1ea20 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 08/04/2025 02:16:43.889 7f0328d680f0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 08/04/2025 02:16:43.890 7f0328d680f0 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 08/04/2025 02:16:43.890 7f0328d680f0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 08/04/2025 02:16:43.891 7f0328d680f0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 08/04/2025 02:16:43.891 7f0328d680f0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 08/04/2025 02:16:43.892 7f0328d680f0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 08/04/2025 02:16:43.892 7f0328d68f60 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 08/04/2025 02:16:43.897 7f0328d68f60 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 08/04/2025 02:16:43.983 7f032b138f48 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/04/2025 02:16:49.016 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3749-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=16420328, partition_space_available_1k=64670156, percentage_partition_space_used=20, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2029964, cpu_share=0.13, mem_res=2001752, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=81224, cpu_share=0.0, mem_res=78104, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52852, cpu_share=0.02, mem_res=40352, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=8416644, free=7289112, buffers=711020, cached=16248336, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.43, five_min_avg=2.47, fifteen_min_avg=1.53, cpu_share=3.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 08/04/2025 02:16:53.863 7f032af1ea20 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001439 08/04/2025 02:16:53.863 7f032af1ea20 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 08/04/2025 02:16:53.864 7f032c20bc18 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 08/04/2025 02:16:53.864 7f032c20bc18 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 3 08/04/2025 02:16:53.864 7f032c20bc18 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 08/04/2025 02:16:53.864 7f032c20bc18 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 08/04/2025 02:16:53.864 7f032c20bc18 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 08/04/2025 02:16:53.865 7f03273b2630 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 08/04/2025 02:16:53.865 7f03273b2630 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 08/04/2025 02:17:42.037 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 08/04/2025 02:17:44.573 7f032b138f48 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/04/2025 02:17:49.608 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3749-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17694160, partition_space_available_1k=63396324, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2015044, cpu_share=0.06, mem_res=1997212, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167860, cpu_share=0.04, mem_res=164464, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42976, cpu_share=0.03, mem_res=40412, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=9568188, free=19699052, buffers=747892, cached=2649980, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.57, five_min_avg=3.0, fifteen_min_avg=1.78, cpu_share=5.38, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 08/04/2025 02:18:41.798 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 08/04/2025 02:18:44.121 7f032b138f48 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/04/2025 02:18:49.169 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3749-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17819840, partition_space_available_1k=63270644, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2033860, cpu_share=0.05, mem_res=2007832, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=168184, cpu_share=0.0, mem_res=164788, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=43348, cpu_share=0.03, mem_res=40412, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=8889188, free=20226756, buffers=751704, cached=2797464, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.23, five_min_avg=3.51, fifteen_min_avg=2.03, cpu_share=5.12, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 08/04/2025 02:19:39.373 7f032863c240 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 08/04/2025 02:19:41.676 7f032866b390 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 08/04/2025 02:19:41.792 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 08/04/2025 02:19:44.204 7f032b138f48 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/04/2025 02:19:49.242 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3749-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=18201608, partition_space_available_1k=62888876, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2026484, cpu_share=0.06, mem_res=1999512, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=168608, cpu_share=0.0, mem_res=165204, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=43064, cpu_share=0.03, mem_res=40460, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=9573724, free=19390472, buffers=757772, cached=2943144, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.39, five_min_avg=3.28, fifteen_min_avg=2.04, cpu_share=2.88, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 08/04/2025 02:20:41.836 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 08/04/2025 02:20:44.144 7f032b138f48 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/04/2025 02:20:49.179 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3749-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19368400, partition_space_available_1k=61722084, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2028708, cpu_share=0.06, mem_res=2001640, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=172932, cpu_share=0.0, mem_res=169520, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=43088, cpu_share=0.03, mem_res=40488, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=9948520, free=17708540, buffers=788024, cached=4220028, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.28, five_min_avg=3.28, fifteen_min_avg=2.12, cpu_share=4.12, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 08/04/2025 02:21:41.830 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 08/04/2025 02:21:44.208 7f032b138f48 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/04/2025 02:21:49.248 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3749-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19413656, partition_space_available_1k=61676828, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2039464, cpu_share=0.16, mem_res=2011592, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=175004, cpu_share=0.0, mem_res=171572, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=43092, cpu_share=0.03, mem_res=40488, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=10185664, free=17390180, buffers=788772, cached=4300496, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.71, five_min_avg=3.94, fifteen_min_avg=2.41, cpu_share=4.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 08/04/2025 02:22:42.116 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 08/04/2025 02:22:44.451 7f032b138f48 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/04/2025 02:22:49.485 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3749-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19529420, partition_space_available_1k=61561064, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2043148, cpu_share=0.17, mem_res=2014668, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=179036, cpu_share=0.01, mem_res=175568, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=43096, cpu_share=0.03, mem_res=40488, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=10877832, free=16575728, buffers=788860, cached=4422692, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.28, five_min_avg=3.87, fifteen_min_avg=2.49, cpu_share=5.88, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 08/04/2025 02:23:41.812 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 08/04/2025 02:23:44.246 7f032b138f48 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/04/2025 02:23:49.292 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3749-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19558468, partition_space_available_1k=61532016, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2054432, cpu_share=0.15, mem_res=2025344, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=191076, cpu_share=0.02, mem_res=187548, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=43108, cpu_share=0.03, mem_res=40488, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=10373336, free=17032988, buffers=789232, cached=4469556, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.63, five_min_avg=3.79, fifteen_min_avg=2.55, cpu_share=4.62, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 08/04/2025 02:24:41.826 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 08/04/2025 02:24:44.665 7f032b138f48 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/04/2025 02:24:49.706 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3749-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19589724, partition_space_available_1k=61500760, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2058896, cpu_share=0.13, mem_res=2028712, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=192388, cpu_share=0.01, mem_res=188824, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=43112, cpu_share=0.03, mem_res=40488, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=10975820, free=16395420, buffers=789440, cached=4504432, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.23, five_min_avg=3.67, fifteen_min_avg=2.59, cpu_share=5.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 08/04/2025 02:25:41.816 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 08/04/2025 02:25:44.165 7f032b138f48 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/04/2025 02:25:49.202 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3749-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19677768, partition_space_available_1k=61412716, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2053652, cpu_share=0.09, mem_res=2023308, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=194308, cpu_share=0.0, mem_res=190724, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=43116, cpu_share=0.03, mem_res=40488, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=11548968, free=15770396, buffers=789500, cached=4556248, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.49, five_min_avg=3.67, fifteen_min_avg=2.65, cpu_share=4.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 08/04/2025 02:26:41.832 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 08/04/2025 02:26:44.068 7f032b138f48 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/04/2025 02:26:49.106 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3749-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19676536, partition_space_available_1k=61413948, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2056748, cpu_share=0.1, mem_res=2026020, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=196072, cpu_share=0.0, mem_res=192472, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=43120, cpu_share=0.03, mem_res=40488, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=11062904, free=16231072, buffers=789512, cached=4581624, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.09, five_min_avg=3.52, fifteen_min_avg=2.66, cpu_share=4.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 08/04/2025 02:27:41.855 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 08/04/2025 02:27:43.910 7f032b138f48 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/04/2025 02:27:48.949 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3749-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19697980, partition_space_available_1k=61392504, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2063076, cpu_share=0.08, mem_res=2032136, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=197828, cpu_share=0.0, mem_res=194204, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=43120, cpu_share=0.03, mem_res=40488, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=10750900, free=16505240, buffers=789780, cached=4619192, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.41, five_min_avg=3.57, fifteen_min_avg=2.74, cpu_share=4.38, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 08/04/2025 02:28:41.839 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 08/04/2025 02:28:44.183 7f032b138f48 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/04/2025 02:28:49.230 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3749-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19720984, partition_space_available_1k=61369500, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2056652, cpu_share=0.08, mem_res=2025560, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=199276, cpu_share=0.0, mem_res=195644, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=43824, cpu_share=0.03, mem_res=41184, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=11407776, free=15822384, buffers=789832, cached=4645120, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.47, five_min_avg=3.59, fifteen_min_avg=2.8, cpu_share=4.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 08/04/2025 02:29:41.860 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 08/04/2025 02:29:44.123 7f032b138f48 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/04/2025 02:29:49.160 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3749-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19813948, partition_space_available_1k=61276536, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2060276, cpu_share=0.08, mem_res=2027964, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=200548, cpu_share=0.0, mem_res=196912, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=43824, cpu_share=0.03, mem_res=41184, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=10503780, free=16628280, buffers=789908, cached=4743144, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.85, five_min_avg=3.64, fifteen_min_avg=2.87, cpu_share=4.62, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 08/04/2025 02:30:41.874 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 08/04/2025 02:30:44.211 7f032b138f48 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/04/2025 02:30:49.246 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3749-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19820704, partition_space_available_1k=61269780, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2065504, cpu_share=0.08, mem_res=2033168, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=202116, cpu_share=0.0, mem_res=198472, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=43832, cpu_share=0.03, mem_res=41184, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=10540568, free=16582064, buffers=789912, cached=4752568, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.87, five_min_avg=3.43, fifteen_min_avg=2.84, cpu_share=3.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 08/04/2025 02:31:41.870 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 08/04/2025 02:31:43.967 7f032b138f48 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/04/2025 02:31:44.519 7f03273b2c18 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 08/04/2025 02:31:49.000 7f032b138f48 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3749-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19884892, partition_space_available_1k=61205592, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2061196, cpu_share=0.07, mem_res=2028660, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=202996, cpu_share=0.0, mem_res=199340, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=43832, cpu_share=0.03, mem_res=41184, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=10350148, free=16576060, buffers=792748, cached=4946156, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.33, five_min_avg=3.2, fifteen_min_avg=2.8, cpu_share=3.12, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')