09/01/2025 01:57:17.026 7f8fae9c6fc0 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/01/2025 01:57:17.026 7f8fae9c6fc0 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/01/2025 01:57:17.026 7f8fae9c6fc0 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/01/2025 01:57:17.026 7f8fa9811040 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 09/01/2025 01:57:17.026 7f8fa9811040 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:57:17.027 7f8fa9811040 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/01/2025 01:57:17.027 7f8fa9811040 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/01/2025 01:57:17.027 7f8fa9811040 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:17.027 7f8fa9811040 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/01/2025 01:57:17.027 7f8fa9811040 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:17.027 7f8fa9811040 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:17.027 7f8fa9811040 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 start_time = 1756691837027098 collector_name = collector_ip = collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/01/2025 01:57:17.027 7f8fa9811040 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:17.050 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.56:5673//'] 09/01/2025 01:57:17.058 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 09/01/2025 01:57:17.059 7f8fa98119a0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:17.059 7f8fa98119a0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 09/01/2025 01:57:17.059 7f8fa98119a0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:17.059 7f8fa98119a0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:17.059 7f8fa98119a0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 09/01/2025 01:57:17.060 7f8fa98119a0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:17.092 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/01/2025 01:57:17.093 7f8fabe8d4a0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:17.093 7f8fabe8d4a0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/01/2025 01:57:17.093 7f8fabe8d4a0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:17.103 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 09/01/2025 01:57:17.103 7f8fa95aa0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:17.104 7f8fa95aa0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 09/01/2025 01:57:17.104 7f8fa95aa0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:17.106 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/01/2025 01:57:17.106 7f8fa95aa0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:17.106 7f8fa95aa0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/01/2025 01:57:17.107 7f8fa95aa0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:17.117 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 09/01/2025 01:57:17.117 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 09/01/2025 01:57:17.117 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: initializing cassandra cluster 09/01/2025 01:57:17.118 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/01/2025 01:57:17.118 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 09/01/2025 01:57:17.118 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 09/01/2025 01:57:17.118 7f8fa95aa680 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 09/01/2025 01:57:17.118 7f8fa95aa680 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 09/01/2025 01:57:17.119 7f8fa95aa900 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:17.119 7f8fa95aa900 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:57:17.119 7f8fa95aa900 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:17.120 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/01/2025 01:57:17.124 7f8fa95aa540 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 09/01/2025 01:57:17.124 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 09/01/2025 01:57:17.125 7f8fa95aa680 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 09/01/2025 01:57:17.161 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/01/2025 01:57:17.163 7f8fa95a6680 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:17.164 7f8fa95a6680 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/01/2025 01:57:17.164 7f8fa95a6680 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:17.169 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 09/01/2025 01:57:17.170 7f8fa95a65e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:17.170 7f8fa95a65e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:57:17.170 7f8fa95a65e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:17.176 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 09/01/2025 01:57:17.189 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 09/01/2025 01:57:17.421 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:57:17.423 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: table 'obj_shared_table', created 09/01/2025 01:57:17.427 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:57:17.729 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:57:17.866 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: table 'obj_shared_table' fixed with properties {'gc_grace_seconds': 864000} 09/01/2025 01:57:17.872 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:57:17.873 7f8fa99a35e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:17.873 7f8fa99a35e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/01/2025 01:57:17.873 7f8fa99a35e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:17.904 7f8fa95aa7c0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:17.905 7f8fa95aa7c0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:Zookeeper[] connection down >>, ] >> 09/01/2025 01:57:17.905 7f8fa95aa7c0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:17.918 7f8fa99a3ae0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:17.919 7f8fa99a3ae0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/01/2025 01:57:17.919 7f8fa99a3ae0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:17.922 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 09/01/2025 01:57:17.923 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 09/01/2025 01:57:17.924 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 09/01/2025 01:57:17.925 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3993-1. 09/01/2025 01:57:17.931 7f8fa95aa540 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 09/01/2025 01:57:17.945 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3993-1.'] 09/01/2025 01:57:17.953 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 09/01/2025 01:57:17.957 7f8fa95aa540 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 09/01/2025 01:57:17.981 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3993-1.'] 09/01/2025 01:57:18.449 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Fetching all containers 09/01/2025 01:57:18.651 7f8fae9c6fc0 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 09/01/2025 01:57:18.977 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3993-1..config_queue 09/01/2025 01:57:18.978 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3993-1..tftp_queue 09/01/2025 01:57:18.983 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:18.991 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 09/01/2025 01:57:18.996 7f8fa95aa540 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 09/01/2025 01:57:19.017 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3993-1.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3993-1..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3993-1..tftp_queue'] 09/01/2025 01:57:19.479 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Process 109 prepared to run in Full mode... 09/01/2025 01:57:19.479 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 09/01/2025 01:57:19.479 7f8fae9c6fc0 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 09/01/2025 01:57:19.485 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/local/lib/python3.9/site-packages/cfgm_common/uve/config_updater] to home page 09/01/2025 01:57:19.485 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 09/01/2025 01:57:19.487 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/local/lib/python3.9/site-packages/cfgm_common/uve/greenlets] to home page 09/01/2025 01:57:19.487 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 09/01/2025 01:57:19.489 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/local/lib/python3.9/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 09/01/2025 01:57:19.489 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 09/01/2025 01:57:19.496 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/local/lib/python3.9/site-packages/device_manager/sandesh/dm_introspect] to home page 09/01/2025 01:57:19.496 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 09/01/2025 01:57:19.496 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 09/01/2025 01:57:19.496 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 09/01/2025 01:57:19.496 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 09/01/2025 01:57:19.497 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 09/01/2025 01:57:19.497 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 09/01/2025 01:57:19.497 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 09/01/2025 01:57:19.497 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 09/01/2025 01:57:19.497 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 09/01/2025 01:57:19.497 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 09/01/2025 01:57:19.497 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 09/01/2025 01:57:19.497 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 09/01/2025 01:57:19.500 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/local/lib/python3.9/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 09/01/2025 01:57:19.500 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 09/01/2025 01:57:19.500 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 09/01/2025 01:57:19.500 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 09/01/2025 01:57:19.501 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/local/lib/python3.9/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 09/01/2025 01:57:19.501 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 09/01/2025 01:57:19.501 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 09/01/2025 01:57:19.502 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/local/lib/python3.9/site-packages/pysandesh/gen_py/sandesh_trace] to home page 09/01/2025 01:57:19.502 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 09/01/2025 01:57:19.502 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 09/01/2025 01:57:19.502 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 09/01/2025 01:57:19.502 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 09/01/2025 01:57:19.502 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 09/01/2025 01:57:19.504 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/local/lib/python3.9/site-packages/pysandesh/gen_py/sandesh_uve] to home page 09/01/2025 01:57:19.504 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 09/01/2025 01:57:19.504 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 09/01/2025 01:57:19.505 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 09/01/2025 01:57:19.505 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 09/01/2025 01:57:19.505 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 09/01/2025 01:57:19.505 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 09/01/2025 01:57:19.505 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 09/01/2025 01:57:19.505 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 09/01/2025 01:57:19.505 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 09/01/2025 01:57:19.506 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 09/01/2025 01:57:19.506 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 09/01/2025 01:57:19.506 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 09/01/2025 01:57:19.506 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 09/01/2025 01:57:19.506 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 09/01/2025 01:57:19.509 7f8fae9c6fc0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 09/01/2025 01:57:19.763 7f8fa95aaae0 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 09/01/2025 01:57:19.763 7f8fa95aaae0 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 09/01/2025 01:57:19.763 7f8fa86b6360 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:19.764 7f8fa86b6360 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[none to Idle on EvStart], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 09/01/2025 01:57:19.764 7f8fa86b6360 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:19.765 7f8fa86b6360 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:19.766 7f8fa86b6360 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[none to Idle on EvStart], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 09/01/2025 01:57:19.766 7f8fa86b6360 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:20.020 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:20.026 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 09/01/2025 01:57:20.030 7f8fa95aa540 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 09/01/2025 01:57:20.039 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3993-1.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3993-1..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3993-1..tftp_queue', 'device_manager_ztp.ztp_queue'] 09/01/2025 01:57:21.028 7f8fa99a3900 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:21.030 7f8fa99a3900 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:21.030 7f8fa99a3900 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:21.030 7f8fa99a3900 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:21.031 7f8fa99a3900 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 09/01/2025 01:57:21.031 7f8fa99a3900 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:21.031 7f8fa99a3900 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:21.032 7f8fa99a3900 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:21.032 7f8fa99a3900 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:21.032 7f8fa99a3900 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 09/01/2025 01:57:21.033 7f8fa99a3900 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:21.033 7f8fa99a3900 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:21.033 7f8fa99a3900 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:21.033 7f8fa99a3900 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:21.042 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:22.043 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:22.782 7f8fae9c6fc0 [contrail-device-manager] [ERROR]: predef payloads file does not exist: /opt/contrail/fabric_ansible_playbooks/conf/predef_payloads.json 09/01/2025 01:57:22.782 7f8fae9c6fc0 [contrail-device-manager] [ERROR]: Unable to load init data 09/01/2025 01:57:22.782 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 09/01/2025 01:57:22.782 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: initializing cassandra cluster 09/01/2025 01:57:22.782 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/01/2025 01:57:22.783 7f8faa2f37c0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:22.783 7f8faa2f37c0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:22.783 7f8faa2f37c0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:22.784 7f8faa2f37c0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:22.784 7f8faa2f37c0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:57:22.784 7f8faa2f37c0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:22.786 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/01/2025 01:57:22.820 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/01/2025 01:57:22.822 7f8fa86b6ea0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:22.824 7f8fa86b6ea0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Initializing description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/01/2025 01:57:22.824 7f8fa86b6ea0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:22.830 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 09/01/2025 01:57:22.830 7f8fa86b6ea0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:22.830 7f8fa86b6ea0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:57:22.830 7f8fa86b6ea0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:22.852 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:57:22.915 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace', created 09/01/2025 01:57:22.919 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:57:22.925 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:57:22.977 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None} 09/01/2025 01:57:22.981 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:57:22.981 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 09/01/2025 01:57:23.013 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 09/01/2025 01:57:23.021 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:57:23.044 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:23.244 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table', created 09/01/2025 01:57:23.251 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:57:23.261 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:57:23.400 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table' fixed with properties {'gc_grace_seconds': 864000} 09/01/2025 01:57:23.408 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:57:23.416 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:57:23.586 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table', created 09/01/2025 01:57:23.592 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:57:23.607 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:57:23.849 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table' fixed with properties {'gc_grace_seconds': 864000} 09/01/2025 01:57:23.867 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:57:23.886 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:57:24.047 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:24.223 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table', created 09/01/2025 01:57:24.228 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:57:24.241 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:57:24.413 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table' fixed with properties {'gc_grace_seconds': 864000} 09/01/2025 01:57:24.431 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:57:24.441 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:57:24.652 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table', created 09/01/2025 01:57:24.662 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:57:24.673 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:57:24.796 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table' fixed with properties {'gc_grace_seconds': 864000} 09/01/2025 01:57:24.805 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:57:24.806 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:24.806 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:24.807 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:24.822 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.56:5673//'] 09/01/2025 01:57:24.822 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 09/01/2025 01:57:24.823 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:24.823 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Initializing description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 09/01/2025 01:57:24.823 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:24.823 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:24.823 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 09/01/2025 01:57:24.823 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:24.826 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/01/2025 01:57:24.826 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:24.826 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:24.826 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:24.830 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 09/01/2025 01:57:24.831 7f8fa80c59a0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:24.831 7f8fa80c59a0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 09/01/2025 01:57:24.831 7f8fa80c59a0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:24.834 7f8fae9c6fc0 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/01/2025 01:57:24.834 7f8fa80c59a0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:24.835 7f8fa80c59a0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:24.835 7f8fa80c59a0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:24.835 7f8fa86b67c0 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 09/01/2025 01:57:24.836 7f8fa80fd220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:24.836 7f8fa80fd220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 09/01/2025 01:57:24.836 7f8fa80fd220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:24.838 7f8fa86b67c0 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/01/2025 01:57:24.838 7f8fa80fd400 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:24.838 7f8fa80fd400 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:24.838 7f8fa80fd400 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:24.928 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'analyzer-template'], 'obj_dict': {'display_name': 'analyzer-template', 'fq_name': ['default-domain', 'analyzer-template'], 'id_perms': {'created': '2025-09-01T01:57:24.869959', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:57:24.869959', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13521180333866150676, 'uuid_mslong': 9793958929396417514}}, 'parent_type': 'domain', 'parent_uuid': '29b069c3-005d-4827-93b3-22f4ba94fd44', 'perms2': {'global_access': 0, 'owner': '9ef9ee5db15946d4bf11570dab196c5a', 'owner_access': 7, 'share': [{'tenant': 'domain:29b069c3-005d-4827-93b3-22f4ba94fd44', 'tenant_access': 5}]}, 'service_template_properties': {'availability_zone_enable': False, 'flavor': 'm1.medium', 'image_name': 'analyzer', 'instance_data': None, 'interface_type': [{'service_interface_type': 'left', 'shared_ip': False, 'static_route_enable': False}], 'ordered_interfaces': True, 'service_mode': None, 'service_scaling': False, 'service_type': 'analyzer', 'service_virtualization_type': 'virtual-machine', 'version': 1, 'vrouter_instance_type': None}, 'uuid': '87eb21c1-7370-43ea-bba4-e1e01816eb14'}, 'oper': 'CREATE', 'request-id': 'req-3c8bbc39-dc34-4efc-9dc6-057f1b6d0040', 'type': 'service_template', 'uuid': '87eb21c1-7370-43ea-bba4-e1e01816eb14'} 09/01/2025 01:57:24.930 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'nat-template'], 'obj_dict': {'display_name': 'nat-template', 'fq_name': ['default-domain', 'nat-template'], 'id_perms': {'created': '2025-09-01T01:57:24.905120', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:57:24.905120', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11611091319806425669, 'uuid_mslong': 5332057076780975597}}, 'parent_type': 'domain', 'parent_uuid': '29b069c3-005d-4827-93b3-22f4ba94fd44', 'perms2': {'global_access': 0, 'owner': '9ef9ee5db15946d4bf11570dab196c5a', 'owner_access': 7, 'share': [{'tenant': 'domain:29b069c3-005d-4827-93b3-22f4ba94fd44', 'tenant_access': 5}]}, 'service_template_properties': {'availability_zone_enable': False, 'flavor': 'm1.medium', 'image_name': 'analyzer', 'instance_data': None, 'interface_type': [{'service_interface_type': 'management', 'shared_ip': False, 'static_route_enable': False}, {'service_interface_type': 'left', 'shared_ip': False, 'static_route_enable': False}, {'service_interface_type': 'right', 'shared_ip': False, 'static_route_enable': False}], 'ordered_interfaces': True, 'service_mode': 'in-network-nat', 'service_scaling': False, 'service_type': 'firewall', 'service_virtualization_type': 'virtual-machine', 'version': 1, 'vrouter_instance_type': None}, 'uuid': '49ff45a9-2fa3-45ed-a122-e20983aefa45'}, 'oper': 'CREATE', 'request-id': 'req-04b84baf-533d-4ec6-a4cc-3aa2efb8f065', 'type': 'service_template', 'uuid': '49ff45a9-2fa3-45ed-a122-e20983aefa45'} 09/01/2025 01:57:24.931 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', '__no_rule__', 'egress-access-control-list'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [], 'dynamic': None}, 'display_name': 'egress-access-control-list', 'fq_name': ['default-domain', 'default-project', '__no_rule__', 'egress-access-control-list'], 'id_perms': {'created': '2025-09-01T01:57:24.909576', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:57:24.909576', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12868985776250880682, 'uuid_mslong': 2405709873795646673}}, 'parent_type': 'security-group', 'parent_uuid': '620e0e4f-8700-4660-910a-f945774113ec', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '2162cc62-5c3d-44d1-b297-d26174fd02aa'}, 'oper': 'CREATE', 'request-id': 'req-49824fae-f844-44b8-acce-680508088a94', 'type': 'access_control_list', 'uuid': '2162cc62-5c3d-44d1-b297-d26174fd02aa'} 09/01/2025 01:57:24.996 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'netns-snat-template'], 'obj_dict': {'display_name': 'netns-snat-template', 'fq_name': ['default-domain', 'netns-snat-template'], 'id_perms': {'created': '2025-09-01T01:57:24.988887', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:57:24.988887', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13127125860065436648, 'uuid_mslong': 9297780873956968404}}, 'parent_type': 'domain', 'parent_uuid': '29b069c3-005d-4827-93b3-22f4ba94fd44', 'perms2': {'global_access': 0, 'owner': '9ef9ee5db15946d4bf11570dab196c5a', 'owner_access': 7, 'share': [{'tenant': 'domain:29b069c3-005d-4827-93b3-22f4ba94fd44', 'tenant_access': 5}]}, 'service_template_properties': {'availability_zone_enable': False, 'flavor': None, 'image_name': None, 'instance_data': None, 'interface_type': [{'service_interface_type': 'right', 'shared_ip': True, 'static_route_enable': False}, {'service_interface_type': 'left', 'shared_ip': True, 'static_route_enable': False}], 'ordered_interfaces': True, 'service_mode': 'in-network-nat', 'service_scaling': True, 'service_type': 'source-nat', 'service_virtualization_type': 'network-namespace', 'version': 1, 'vrouter_instance_type': None}, 'uuid': '81085a71-6fa0-47d4-b62c-eb6b530b63e8'}, 'oper': 'CREATE', 'request-id': 'req-e74f30f8-6872-4452-828c-6921aa761df1', 'type': 'service_template', 'uuid': '81085a71-6fa0-47d4-b62c-eb6b530b63e8'} 09/01/2025 01:57:25.031 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:25.032 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:25.032 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:25.032 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:25.033 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:25.033 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:25.033 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:25.033 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:25.033 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:25.033 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:25.034 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:25.034 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:25.034 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:25.034 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:25.040 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['target:64512:8000000'], 'obj_dict': {'display_name': 'target:64512:8000000', 'fq_name': ['target:64512:8000000'], 'id_perms': {'created': '2025-09-01T01:57:25.032076', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:57:25.032076', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13445005882208166071, 'uuid_mslong': 3179637496494903218}}, 'perms2': {'global_access': 0, 'owner': '9ef9ee5db15946d4bf11570dab196c5a', 'owner_access': 7, 'share': []}, 'uuid': '2c205774-e517-4fb2-ba96-419d7c0e20b7'}, 'oper': 'CREATE', 'request-id': 'req-62e5a752-f19f-4215-b343-d75b199558ab', 'type': 'route_target', 'uuid': '2c205774-e517-4fb2-ba96-419d7c0e20b7'} 09/01/2025 01:57:25.048 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:25.066 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'dci-network', '__default__'], 'oper': 'UPDATE', 'request-id': 'req-e5e94a13-1fc3-4c2c-9f3d-8a05f4552f88', 'type': 'routing_instance', 'uuid': 'e539740b-2375-4404-b14f-3d22969f17f7'} 09/01/2025 01:57:25.083 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'haproxy-loadbalancer-template'], 'obj_dict': {'display_name': 'haproxy-loadbalancer-template', 'fq_name': ['default-domain', 'haproxy-loadbalancer-template'], 'id_perms': {'created': '2025-09-01T01:57:25.075674', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:57:25.075674', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9521065237204121824, 'uuid_mslong': 5101078152735048275}}, 'parent_type': 'domain', 'parent_uuid': '29b069c3-005d-4827-93b3-22f4ba94fd44', 'perms2': {'global_access': 0, 'owner': '9ef9ee5db15946d4bf11570dab196c5a', 'owner_access': 7, 'share': [{'tenant': 'domain:29b069c3-005d-4827-93b3-22f4ba94fd44', 'tenant_access': 5}]}, 'service_template_properties': {'availability_zone_enable': False, 'flavor': None, 'image_name': None, 'instance_data': None, 'interface_type': [{'service_interface_type': 'right', 'shared_ip': True, 'static_route_enable': False}, {'service_interface_type': 'left', 'shared_ip': True, 'static_route_enable': False}], 'ordered_interfaces': True, 'service_mode': 'in-network-nat', 'service_scaling': True, 'service_type': 'loadbalancer', 'service_virtualization_type': 'network-namespace', 'version': 1, 'vrouter_instance_type': None}, 'uuid': '46caab8d-a147-4253-8421-9e6372df38e0'}, 'oper': 'CREATE', 'request-id': 'req-24055c1d-0d35-48a4-b344-c5fe0f077328', 'type': 'service_template', 'uuid': '46caab8d-a147-4253-8421-9e6372df38e0'} 09/01/2025 01:57:25.173 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'docker-template'], 'obj_dict': {'display_name': 'docker-template', 'fq_name': ['default-domain', 'docker-template'], 'id_perms': {'created': '2025-09-01T01:57:25.159671', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:57:25.159671', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11139179131399977529, 'uuid_mslong': 8789539564019075591}}, 'parent_type': 'domain', 'parent_uuid': '29b069c3-005d-4827-93b3-22f4ba94fd44', 'perms2': {'global_access': 0, 'owner': '9ef9ee5db15946d4bf11570dab196c5a', 'owner_access': 7, 'share': [{'tenant': 'domain:29b069c3-005d-4827-93b3-22f4ba94fd44', 'tenant_access': 5}]}, 'service_template_properties': {'availability_zone_enable': False, 'flavor': None, 'image_name': 'ubuntu', 'instance_data': '{"command":"/bin/bash"}', 'interface_type': [{'service_interface_type': 'management', 'shared_ip': False, 'static_route_enable': False}, {'service_interface_type': 'left', 'shared_ip': False, 'static_route_enable': False}, {'service_interface_type': 'right', 'shared_ip': False, 'static_route_enable': False}], 'ordered_interfaces': True, 'service_mode': 'transparent', 'service_scaling': False, 'service_type': 'firewall', 'service_virtualization_type': 'vrouter-instance', 'version': 1, 'vrouter_instance_type': 'docker'}, 'uuid': '79fab7aa-1c99-4a07-9a96-5066f08a3a39'}, 'oper': 'CREATE', 'request-id': 'req-a15d2453-ecec-4c90-b53a-1fbff95e0c87', 'type': 'service_template', 'uuid': '79fab7aa-1c99-4a07-9a96-5066f08a3a39'} 09/01/2025 01:57:25.180 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['target:64512:8000001'], 'obj_dict': {'display_name': 'target:64512:8000001', 'fq_name': ['target:64512:8000001'], 'id_perms': {'created': '2025-09-01T01:57:25.162274', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:57:25.162274', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13507925698563971393, 'uuid_mslong': 3087026659819275195}}, 'perms2': {'global_access': 0, 'owner': '9ef9ee5db15946d4bf11570dab196c5a', 'owner_access': 7, 'share': []}, 'uuid': '2ad75264-2eb0-43bb-bb75-cadad3469941'}, 'oper': 'CREATE', 'request-id': 'req-00ff42a2-3e12-4da3-8957-ea788e904778', 'type': 'route_target', 'uuid': '2ad75264-2eb0-43bb-bb75-cadad3469941'} 09/01/2025 01:57:25.238 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'dci-network', 'dci-network'], 'oper': 'UPDATE', 'request-id': 'req-0ae7a961-afa0-41a0-8773-c73ae67701ea', 'type': 'routing_instance', 'uuid': '8c7a11ed-d168-41be-b4f9-cd3a9b2f8025'} 09/01/2025 01:57:25.331 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['target:64512:8000002'], 'obj_dict': {'display_name': 'target:64512:8000002', 'fq_name': ['target:64512:8000002'], 'id_perms': {'created': '2025-09-01T01:57:25.321933', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:57:25.321933', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10854995415371099679, 'uuid_mslong': 3718999023607695746}}, 'perms2': {'global_access': 0, 'owner': '9ef9ee5db15946d4bf11570dab196c5a', 'owner_access': 7, 'share': []}, 'uuid': '339c89e8-6a29-4982-96a4-b0d16bb02e1f'}, 'oper': 'CREATE', 'request-id': 'req-21ed0f20-9ef0-49ea-b63a-d952189ced06', 'type': 'route_target', 'uuid': '339c89e8-6a29-4982-96a4-b0d16bb02e1f'} 09/01/2025 01:57:25.399 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'default-virtual-network', 'default-virtual-network'], 'oper': 'UPDATE', 'request-id': 'req-c36cfde0-2fa3-4f19-920d-5e7ab8ac023e', 'type': 'routing_instance', 'uuid': 'c5187ec2-16a9-4faa-b7ba-5b40f765ed82'} 09/01/2025 01:57:25.479 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['target:64512:8000003'], 'obj_dict': {'display_name': 'target:64512:8000003', 'fq_name': ['target:64512:8000003'], 'id_perms': {'created': '2025-09-01T01:57:25.471548', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:57:25.471548', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9247724033477844785, 'uuid_mslong': 6465635102301045560}}, 'perms2': {'global_access': 0, 'owner': '9ef9ee5db15946d4bf11570dab196c5a', 'owner_access': 7, 'share': []}, 'uuid': '59ba8cde-7ff3-4338-8056-84030c4a1f31'}, 'oper': 'CREATE', 'request-id': 'req-65150cd0-0e7a-46ef-a203-0daabcc51b27', 'type': 'route_target', 'uuid': '59ba8cde-7ff3-4338-8056-84030c4a1f31'} 09/01/2025 01:57:25.546 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-6be96408-0b1d-4ad3-b461-d00a13833cec', 'type': 'routing_instance', 'uuid': '231b437c-f8ca-4984-8584-b66c8b6969d7'} 09/01/2025 01:57:25.713 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [{'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'default-domain:default-project:ip-fabric'}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': None, 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'default-domain:default-project:ip-fabric'}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '00000000-0000-0000-0000-100000000001'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'deny'}, 'direction': '<>', 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'any'}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': None, 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'default-domain:default-project:ip-fabric'}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '00000000-0000-0000-0000-100000000002'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'any'}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': None, 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'any'}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '00000000-0000-0000-0000-100000000001'}], 'dynamic': False}, 'display_name': 'ip-fabric', 'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'id_perms': {'created': '2025-09-01T01:57:25.659948', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:57:25.659948', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9366827408060485335, 'uuid_mslong': 7610566125744046283}}, 'parent_type': 'virtual-network', 'parent_uuid': 'c39bc5a7-3ad4-482b-9be0-1adbb0ac1242', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '699e2991-a098-40cb-81fd-a7e669d68ed7'}, 'oper': 'CREATE', 'request-id': 'req-2a764eae-e604-4a32-8250-fec835f9fa09', 'type': 'access_control_list', 'uuid': '699e2991-a098-40cb-81fd-a7e669d68ed7'} 09/01/2025 01:57:25.784 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', '__no_rule__', 'ingress-access-control-list'], 'oper': 'UPDATE', 'request-id': 'req-dccc92b7-b6c9-4f5a-beea-2caa11556305', 'type': 'access_control_list', 'uuid': 'dec23a1c-f6fe-47d9-8ea0-e6286b733f0a'} 09/01/2025 01:57:25.852 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', '__no_rule__', 'egress-access-control-list'], 'oper': 'UPDATE', 'request-id': 'req-eb120720-53d3-4802-9f39-e69ba89416d2', 'type': 'access_control_list', 'uuid': '2162cc62-5c3d-44d1-b297-d26174fd02aa'} 09/01/2025 01:57:26.049 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:27.051 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:28.052 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:29.033 7f8fa80fd5e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:29.035 7f8fa80fd5e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:29.035 7f8fa80fd5e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:29.036 7f8fa80fd5e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:29.036 7f8fa80fd5e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:29.036 7f8fa80fd5e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:29.036 7f8fa80fd5e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:29.037 7f8fa80fd5e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:29.037 7f8fa80fd5e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:29.038 7f8fa80fd5e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:29.038 7f8fa80fd5e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:29.038 7f8fa80fd5e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:29.038 7f8fa80fd5e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:29.038 7f8fa80fd5e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:29.054 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:29.279 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'default-global-vrouter-config'], 'oper': 'UPDATE', 'request-id': 'req-cab338b1-5405-4a03-a3a6-05c2a292d4be', 'type': 'global_vrouter_config', 'uuid': '08473562-4927-44da-a90c-47622528dc25'} 09/01/2025 01:57:30.056 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:30.069 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-conf-elements-inconsistency'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'ContrailConfig.elements', 'operand2': {'json_value': '1'}, 'operation': 'size=='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-conf-elements-inconsistency', 'fq_name': ['default-global-system-config', 'system-defined-conf-elements-inconsistency'], 'id_perms': {'created': '2025-09-01T01:57:30.054199', 'creator': None, 'description': 'ContrailConfig has a list of ' 'elements - it should be a dict. ' 'Controllers have inconsistency in ' 'their data.', 'enable': True, 'last_modified': '2025-09-01T01:57:30.054199', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9464576858265937879, 'uuid_mslong': 3056869154938963818}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '2a6c2e4c-e8fc-476a-8358-ee81d252e7d7', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-b2bae1b0-7f80-473e-ab04-39c4345dfca8', 'type': 'alarm', 'uuid': '2a6c2e4c-e8fc-476a-8358-ee81d252e7d7'} 09/01/2025 01:57:30.117 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-address-mismatch-compute'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'ContrailConfig.elements.virtual_router_ip_address', 'operand2': {'uve_attribute': 'VrouterAgent.control_ip'}, 'operation': '!='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-address-mismatch-compute', 'fq_name': ['default-global-system-config', 'system-defined-address-mismatch-compute'], 'id_perms': {'created': '2025-09-01T01:57:30.104070', 'creator': None, 'description': 'Compute Node IP Address mismatch.', 'enable': True, 'last_modified': '2025-09-01T01:57:30.104070', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9683332200319043821, 'uuid_mslong': 1251073803591502090}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '115cb4fe-f2de-490a-8662-1b54ea1e48ed', 'uve_keys': {'uve_key': ['vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-5dc330d9-03e7-43aa-a461-4faac79b70b6', 'type': 'alarm', 'uuid': '115cb4fe-f2de-490a-8662-1b54ea1e48ed'} 09/01/2025 01:57:30.202 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-address-mismatch-control'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'ContrailConfig.elements.bgp_router_parameters.address', 'operand2': {'uve_attribute': 'BgpRouterState.bgp_router_ip_list'}, 'operation': 'not ' 'in'}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-address-mismatch-control', 'fq_name': ['default-global-system-config', 'system-defined-address-mismatch-control'], 'id_perms': {'created': '2025-09-01T01:57:30.193851', 'creator': None, 'description': 'Control Node IP Address mismatch.', 'enable': True, 'last_modified': '2025-09-01T01:57:30.193851', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10830535933819907698, 'uuid_mslong': 3553504101098670032}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '31509527-15e1-47d0-964d-cb0c96706e72', 'uve_keys': {'uve_key': ['control-node']}}, 'oper': 'CREATE', 'request-id': 'req-b6efde5a-c06e-454f-befa-982ca572b141', 'type': 'alarm', 'uuid': '31509527-15e1-47d0-964d-cb0c96706e72'} 09/01/2025 01:57:30.293 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-bgp-connectivity'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'BgpRouterState.num_up_bgp_peer', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}, {'and_list': [{'operand1': 'BgpRouterState.num_up_bgp_peer', 'operand2': {'uve_attribute': 'BgpRouterState.num_bgp_peer'}, 'operation': '!='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-bgp-connectivity', 'fq_name': ['default-global-system-config', 'system-defined-bgp-connectivity'], 'id_perms': {'created': '2025-09-01T01:57:30.275956', 'creator': None, 'description': 'BGP peer mismatch. Not enough BGP ' 'peers are up.', 'enable': True, 'last_modified': '2025-09-01T01:57:30.275956', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13395938943348192093, 'uuid_mslong': 14326837012404456381}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'c6d3265b-1f49-4bbd-b9e7-ef7e8717335d', 'uve_keys': {'uve_key': ['control-node']}}, 'oper': 'CREATE', 'request-id': 'req-f19c69de-e327-4ad3-8f80-7882fc437403', 'type': 'alarm', 'uuid': 'c6d3265b-1f49-4bbd-b9e7-ef7e8717335d'} 09/01/2025 01:57:30.377 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-conf-incorrect'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'ContrailConfig', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-conf-incorrect', 'fq_name': ['default-global-system-config', 'system-defined-conf-incorrect'], 'id_perms': {'created': '2025-09-01T01:57:30.367088', 'creator': None, 'description': 'ContrailConfig missing or ' 'incorrect. Configuration pushed to ' 'Ifmap as ContrailConfig is ' 'missing/incorrect.', 'enable': True, 'last_modified': '2025-09-01T01:57:30.367088', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10820269243797746688, 'uuid_mslong': 15186166341629398445}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'd2c019d3-f4d0-4dad-9629-518c8aa63400', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-9da8650e-2b6a-4265-91fe-cd164a42b888', 'type': 'alarm', 'uuid': 'd2c019d3-f4d0-4dad-9629-518c8aa63400'} 09/01/2025 01:57:30.460 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-disk-usage-high'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.disk_usage_info.*.percentage_partition_space_used', 'operand2': {'json_value': '[70, ' '90]'}, 'operation': 'range', 'variables': ['NodeStatus.disk_usage_info.__key']}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-disk-usage-high', 'fq_name': ['default-global-system-config', 'system-defined-disk-usage-high'], 'id_perms': {'created': '2025-09-01T01:57:30.447796', 'creator': None, 'description': 'Disk usage crosses high threshold ' 'limit.', 'enable': True, 'last_modified': '2025-09-01T01:57:30.447796', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10351272032287242206, 'uuid_mslong': 6248005955484206403}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '56b56050-2b29-4543-8fa7-1b11e9eab7de', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-885a4a39-2518-482b-bd47-96515700cec2', 'type': 'alarm', 'uuid': '56b56050-2b29-4543-8fa7-1b11e9eab7de'} 09/01/2025 01:57:30.552 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-disk-usage-critical'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.disk_usage_info.*.percentage_partition_space_used', 'operand2': {'json_value': '90'}, 'operation': '>', 'variables': ['NodeStatus.disk_usage_info.__key']}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-disk-usage-critical', 'fq_name': ['default-global-system-config', 'system-defined-disk-usage-critical'], 'id_perms': {'created': '2025-09-01T01:57:30.537335', 'creator': None, 'description': 'Disk usage crosses critical ' 'threshold limit.', 'enable': True, 'last_modified': '2025-09-01T01:57:30.537335', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13600940164448598912, 'uuid_mslong': 4838096181709131050}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '43245ed8-0bf2-4d2a-bcc0-3f04c94d3b80', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-f99e466d-c4b1-4502-8a72-df965eec5c58', 'type': 'alarm', 'uuid': '43245ed8-0bf2-4d2a-bcc0-3f04c94d3b80'} 09/01/2025 01:57:30.640 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-node-status'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-node-status', 'fq_name': ['default-global-system-config', 'system-defined-node-status'], 'id_perms': {'created': '2025-09-01T01:57:30.628902', 'creator': None, 'description': 'Node Failure. NodeStatus UVE not ' 'present.', 'enable': True, 'last_modified': '2025-09-01T01:57:30.628902', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10735481443780481004, 'uuid_mslong': 9153565873247047496}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '7f07ffa9-ff07-4f48-94fc-177e7d554fec', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-9d9d857a-e9be-437d-b6de-a49eb9a09c6f', 'type': 'alarm', 'uuid': '7f07ffa9-ff07-4f48-94fc-177e7d554fec'} 09/01/2025 01:57:30.727 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-partial-sysinfo'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.build_info', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-partial-sysinfo', 'fq_name': ['default-global-system-config', 'system-defined-partial-sysinfo'], 'id_perms': {'created': '2025-09-01T01:57:30.714837', 'creator': None, 'description': 'System Info Incomplete.', 'enable': True, 'last_modified': '2025-09-01T01:57:30.714837', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11150891449519288517, 'uuid_mslong': 14343575021455294730}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'c70e9d7c-8b02-410a-9abf-ecb18189b0c5', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-5a272799-5d50-4f04-8b7d-e4da887cec37', 'type': 'alarm', 'uuid': 'c70e9d7c-8b02-410a-9abf-ecb18189b0c5'} 09/01/2025 01:57:30.818 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-process-connectivity'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.process_status', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}, {'and_list': [{'operand1': 'NodeStatus.process_status.state', 'operand2': {'json_value': '"Functional"'}, 'operation': '!=', 'variables': ['NodeStatus.process_status.module_id', 'NodeStatus.process_status.instance_id']}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-process-connectivity', 'fq_name': ['default-global-system-config', 'system-defined-process-connectivity'], 'id_perms': {'created': '2025-09-01T01:57:30.803175', 'creator': None, 'description': 'Process(es) reporting as ' 'non-functional.', 'enable': True, 'last_modified': '2025-09-01T01:57:30.803175', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10894851833224389702, 'uuid_mslong': 8418205608933870664}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '74d37966-56b4-4c48-9732-4a04d7da6446', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-f13093c2-2e62-4fa4-b04b-790d9294fda9', 'type': 'alarm', 'uuid': '74d37966-56b4-4c48-9732-4a04d7da6446'} 09/01/2025 01:57:30.905 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-process-status'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.process_info', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}, {'and_list': [{'operand1': 'NodeStatus.process_info.process_state', 'operand2': {'json_value': '"PROCESS_STATE_RUNNING"'}, 'operation': '!=', 'variables': ['NodeStatus.process_info.process_name']}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-process-status', 'fq_name': ['default-global-system-config', 'system-defined-process-status'], 'id_perms': {'created': '2025-09-01T01:57:30.892287', 'creator': None, 'description': 'Process Failure.', 'enable': True, 'last_modified': '2025-09-01T01:57:30.892287', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13607781469830687097, 'uuid_mslong': 1493426681964743480}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '14b9b7a7-9c99-4b38-bcd8-8d265266a979', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-24782ce6-af27-43f3-9247-e61e3244ad38', 'type': 'alarm', 'uuid': '14b9b7a7-9c99-4b38-bcd8-8d265266a979'} 09/01/2025 01:57:30.993 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-prouter-connectivity'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'ContrailConfig.elements.virtual_router_refs', 'operand2': {'json_value': 'null'}, 'operation': '!='}, {'operand1': 'ProuterData.connected_agent_list', 'operand2': {'json_value': '1'}, 'operation': 'size!='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-prouter-connectivity', 'fq_name': ['default-global-system-config', 'system-defined-prouter-connectivity'], 'id_perms': {'created': '2025-09-01T01:57:30.980556', 'creator': None, 'description': 'Prouter connectivity to controlling ' 'tor agent does not exist we look ' 'for non-empty value for ' 'connected_agent_list', 'enable': True, 'last_modified': '2025-09-01T01:57:30.980556', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10798366937868962885, 'uuid_mslong': 11696224369911022768}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'a2515618-5dd0-44b0-95db-81848e008c45', 'uve_keys': {'uve_key': ['prouter']}}, 'oper': 'CREATE', 'request-id': 'req-444639b2-25b0-42ae-9634-492d50469689', 'type': 'alarm', 'uuid': 'a2515618-5dd0-44b0-95db-81848e008c45'} 09/01/2025 01:57:31.057 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:31.118 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-prouter-tsn-connectivity'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'ContrailConfig.elements.virtual_router_refs', 'operand2': {'json_value': 'null'}, 'operation': '!='}, {'operand1': 'ProuterData.tsn_agent_list', 'operand2': {'json_value': '1'}, 'operation': 'size!='}, {'operand1': 'ProuterData.gateway_mode', 'operand2': {'json_value': '"SERVER"'}, 'operation': '!='}]}, {'and_list': [{'operand1': 'ContrailConfig.elements.virtual_router_refs', 'operand2': {'json_value': 'null'}, 'operation': '!='}, {'operand1': 'ProuterData.tsn_agent_list', 'operand2': {'json_value': '0'}, 'operation': 'size!='}, {'operand1': 'ProuterData.gateway_mode', 'operand2': {'json_value': '"SERVER"'}, 'operation': '=='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-prouter-tsn-connectivity', 'fq_name': ['default-global-system-config', 'system-defined-prouter-tsn-connectivity'], 'id_perms': {'created': '2025-09-01T01:57:31.058068', 'creator': None, 'description': 'Prouter connectivity to controlling ' 'tsn agent does not exist we look ' 'for non-empty value for ' 'tsn_agent_list', 'enable': True, 'last_modified': '2025-09-01T01:57:31.058068', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9679409398720970998, 'uuid_mslong': 7924194410190816163}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '6df864d4-f68f-43a3-8654-2b907e5740f6', 'uve_keys': {'uve_key': ['prouter']}}, 'oper': 'CREATE', 'request-id': 'req-7169866a-210c-4724-9b3f-4a8aad656db4', 'type': 'alarm', 'uuid': '6df864d4-f68f-43a3-8654-2b907e5740f6'} 09/01/2025 01:57:31.202 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-storage-cluster-state'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'StorageCluster.info_stats.status', 'operand2': {'json_value': '0'}, 'operation': '!=', 'variables': ['StorageCluster.info_stats.health_summary']}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-storage-cluster-state', 'fq_name': ['default-global-system-config', 'system-defined-storage-cluster-state'], 'id_perms': {'created': '2025-09-01T01:57:31.187380', 'creator': None, 'description': 'Storage Cluster warning/errors.', 'enable': True, 'last_modified': '2025-09-01T01:57:31.187380', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10316165217059830525, 'uuid_mslong': 6890667409092331147}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '5fa09180-104f-468b-8f2a-619b87177afd', 'uve_keys': {'uve_key': ['storage-cluster']}}, 'oper': 'CREATE', 'request-id': 'req-6daa7478-8fba-48d9-8729-cfa776384766', 'type': 'alarm', 'uuid': '5fa09180-104f-468b-8f2a-619b87177afd'} 09/01/2025 01:57:31.290 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-vrouter-interface'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'VrouterAgent.down_interface_count', 'operand2': {'json_value': '1'}, 'operation': '>=', 'variables': ['VrouterAgent.error_intf_list', 'VrouterAgent.no_config_intf_list']}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-vrouter-interface', 'fq_name': ['default-global-system-config', 'system-defined-vrouter-interface'], 'id_perms': {'created': '2025-09-01T01:57:31.275962', 'creator': None, 'description': 'Vrouter interface(s) down.', 'enable': True, 'last_modified': '2025-09-01T01:57:31.275962', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11276380950649131098, 'uuid_mslong': 4639482557382018409}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '4062c0ce-0e60-4569-9c7d-c0bad36c505a', 'uve_keys': {'uve_key': ['vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-a1360b9c-e102-4f4d-ba6a-b36ab707f172', 'type': 'alarm', 'uuid': '4062c0ce-0e60-4569-9c7d-c0bad36c505a'} 09/01/2025 01:57:31.384 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-xmpp-connectivity'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'BgpRouterState.num_up_xmpp_peer', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}, {'and_list': [{'operand1': 'BgpRouterState.num_up_xmpp_peer', 'operand2': {'uve_attribute': 'BgpRouterState.num_xmpp_peer'}, 'operation': '!='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-xmpp-connectivity', 'fq_name': ['default-global-system-config', 'system-defined-xmpp-connectivity'], 'id_perms': {'created': '2025-09-01T01:57:31.369979', 'creator': None, 'description': 'XMPP peer mismatch.', 'enable': True, 'last_modified': '2025-09-01T01:57:31.369979', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11934772505184287857, 'uuid_mslong': 5811969849151538316}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '50a843c8-dff5-448c-a5a0-d45c3e9bbc71', 'uve_keys': {'uve_key': ['control-node']}}, 'oper': 'CREATE', 'request-id': 'req-d9a4961b-946b-46bc-a3e9-8e13e1b3565b', 'type': 'alarm', 'uuid': '50a843c8-dff5-448c-a5a0-d45c3e9bbc71'} 09/01/2025 01:57:31.472 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-xmpp-close-reason'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'XmppPeerInfoData.close_reason', 'operand2': {'json_value': 'null'}, 'operation': '!='}, {'operand1': 'XmppPeerInfoData.state_info.state', 'operand2': {'json_value': '"Established"'}, 'operation': '!='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-xmpp-close-reason', 'fq_name': ['default-global-system-config', 'system-defined-xmpp-close-reason'], 'id_perms': {'created': '2025-09-01T01:57:31.455463', 'creator': None, 'description': 'XMPP connection closed towards ' 'peer, ' 'alarm has close reason', 'enable': True, 'last_modified': '2025-09-01T01:57:31.455463', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10114706536922320676, 'uuid_mslong': 968344522386194724}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '0d704036-6d06-4124-8c5e-a8015c938f24', 'uve_keys': {'uve_key': ['xmpp-peer']}}, 'oper': 'CREATE', 'request-id': 'req-8f498e8a-e49f-46ed-ae3b-3f74f6da992d', 'type': 'alarm', 'uuid': '0d704036-6d06-4124-8c5e-a8015c938f24'} 09/01/2025 01:57:31.553 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-core-files'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.all_core_file_list', 'operand2': {'json_value': 'null'}, 'operation': '!='}, {'operand1': 'NodeStatus.all_core_file_list', 'operand2': {'json_value': '0'}, 'operation': 'size!='}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-core-files', 'fq_name': ['default-global-system-config', 'system-defined-core-files'], 'id_perms': {'created': '2025-09-01T01:57:31.544086', 'creator': None, 'description': 'A core file has been generated on ' 'the node.', 'enable': True, 'last_modified': '2025-09-01T01:57:31.544086', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10067795827070312262, 'uuid_mslong': 18329283790498910002}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'fe5eb27e-a803-4f32-8bb7-fef697af9f46', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-1b360d25-4fb4-45e1-8897-c8f7ff7cbe8c', 'type': 'alarm', 'uuid': 'fe5eb27e-a803-4f32-8bb7-fef697af9f46'} 09/01/2025 01:57:31.626 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-pending-cassandra-compaction-tasks'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'CassandraStatusData.cassandra_compaction_task.pending_compaction_tasks', 'operand2': {'json_value': '300'}, 'operation': '>='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-pending-cassandra-compaction-tasks', 'fq_name': ['default-global-system-config', 'system-defined-pending-cassandra-compaction-tasks'], 'id_perms': {'created': '2025-09-01T01:57:31.616211', 'creator': None, 'description': 'Pending compaction tasks in ' 'cassandra crossed the configured ' 'threshold.', 'enable': True, 'last_modified': '2025-09-01T01:57:31.616211', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10239805476498944643, 'uuid_mslong': 11871788326234899637}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'a4c11095-51dc-44b5-8e1b-18d3fbd1b283', 'uve_keys': {'uve_key': ['database-nodeconfig-database-node']}}, 'oper': 'CREATE', 'request-id': 'req-549d826e-6be4-4c14-8a1b-1b49be709170', 'type': 'alarm', 'uuid': 'a4c11095-51dc-44b5-8e1b-18d3fbd1b283'} 09/01/2025 01:57:31.712 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-package-version-mismatch'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.running_package_version', 'operand2': {'uve_attribute': 'NodeStatus.installed_package_version'}, 'operation': '!='}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-package-version-mismatch', 'fq_name': ['default-global-system-config', 'system-defined-package-version-mismatch'], 'id_perms': {'created': '2025-09-01T01:57:31.699790', 'creator': None, 'description': 'There is a mismatch between ' 'installed and running package ' 'version.', 'enable': True, 'last_modified': '2025-09-01T01:57:31.699790', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11974809962785940499, 'uuid_mslong': 18005816803787425937}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'f9e182fb-f082-4891-a62f-123744aa2413', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-b25c7652-604a-42e9-a321-b2e29ed96e83', 'type': 'alarm', 'uuid': 'f9e182fb-f082-4891-a62f-123744aa2413'} 09/01/2025 01:57:31.801 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-vrouter-limit-exceeded'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'VrouterAgent.res_limit', 'operand2': {'json_value': 'true'}, 'operation': '==', 'variables': ['VrouterAgent.vr_limit_exceeded_list']}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-vrouter-limit-exceeded', 'fq_name': ['default-global-system-config', 'system-defined-vrouter-limit-exceeded'], 'id_perms': {'created': '2025-09-01T01:57:31.788067', 'creator': None, 'description': 'Agent resource usage exceeded ' 'configured watermark for resource.', 'enable': True, 'last_modified': '2025-09-01T01:57:31.788067', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9724180224848359593, 'uuid_mslong': 4300958671510458183}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '3bb0131f-d104-4f47-86f3-3a64c1f6f0a9', 'uve_keys': {'uve_key': ['vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-8c140382-5c5e-4272-84e6-3fca9df476a0', 'type': 'alarm', 'uuid': '3bb0131f-d104-4f47-86f3-3a64c1f6f0a9'} 09/01/2025 01:57:31.894 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-vrouter-table-limit-exceeded'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'VrouterAgent.res_table_limit', 'operand2': {'json_value': 'true'}, 'operation': '==', 'variables': ['VrouterAgent.vr_limit_exceeded_list']}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-vrouter-table-limit-exceeded', 'fq_name': ['default-global-system-config', 'system-defined-vrouter-table-limit-exceeded'], 'id_perms': {'created': '2025-09-01T01:57:31.880047', 'creator': None, 'description': 'Agent resource usage exceeded table ' 'size for resource in vrouter.', 'enable': True, 'last_modified': '2025-09-01T01:57:31.880047', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10601837199416426449, 'uuid_mslong': 13230187526605258878}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'b79b1175-7b1c-407e-9321-4ac30dbcffd1', 'uve_keys': {'uve_key': ['vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-b61914d6-6fbc-49d5-971f-11aa0f05175f', 'type': 'alarm', 'uuid': 'b79b1175-7b1c-407e-9321-4ac30dbcffd1'} 09/01/2025 01:57:31.986 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-vrouter-l3mh-physical-interface-link-down'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'VrouterAgent.l3mh_phy_interface_down', 'operand2': {'json_value': 'true'}, 'operation': '==', 'variables': ['VrouterAgent.vr_l3mh_intf_list']}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-vrouter-l3mh-physical-interface-link-down', 'fq_name': ['default-global-system-config', 'system-defined-vrouter-l3mh-physical-interface-link-down'], 'id_perms': {'created': '2025-09-01T01:57:31.973494', 'creator': None, 'description': 'On l3 multihomed compute one or ' 'more physical interfaces are ' 'Inactive.', 'enable': True, 'last_modified': '2025-09-01T01:57:31.973494', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12950168648613563420, 'uuid_mslong': 4019893145475239651}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '37c98784-7b14-4ee3-b3b8-3dc5e2030c1c', 'uve_keys': {'uve_key': ['vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-4e87efb4-b197-4427-94e9-d8685451095c', 'type': 'alarm', 'uuid': '37c98784-7b14-4ee3-b3b8-3dc5e2030c1c'} 09/01/2025 01:57:32.058 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:32.835 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'default-global-vrouter-config'], 'oper': 'UPDATE', 'request-id': 'req-075165f7-503b-4338-92f4-8865a97d36c7', 'type': 'global_vrouter_config', 'uuid': '08473562-4927-44da-a90c-47622528dc25'} 09/01/2025 01:57:33.036 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:33.041 7f8fa80fd720 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:33.042 7f8fa80fd720 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:33.042 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:33.042 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:33.042 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:33.043 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:33.043 7f8fa80fd720 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:33.044 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:33.044 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:33.044 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:33.044 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:33.044 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:33.045 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:33.060 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:33.741 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'default-global-vrouter-config'], 'oper': 'UPDATE', 'request-id': 'req-1d308f62-bab3-411e-adcb-b4572e26b8ed', 'type': 'global_vrouter_config', 'uuid': '08473562-4927-44da-a90c-47622528dc25'} 09/01/2025 01:57:34.061 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:34.659 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-b895319b-a6dc-4cd1-9a53-a15b42800de1', 'type': 'global_system_config', 'uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500'} 09/01/2025 01:57:35.063 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:36.064 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:37.043 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:37.045 7f8fa80fd720 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:37.045 7f8fa80fd720 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:37.045 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:37.045 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:37.046 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:37.046 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:37.046 7f8fa80fd720 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:37.046 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:37.046 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:37.046 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:37.046 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:37.046 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:37.046 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:37.066 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:38.068 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:39.069 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:40.071 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:41.046 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:41.048 7f8fa80fd720 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:41.049 7f8fa80fd720 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:41.049 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:41.049 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:41.049 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:41.050 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:41.050 7f8fa80fd720 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:41.051 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:41.051 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:41.051 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:41.051 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:41.052 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:41.052 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:41.072 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:42.074 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:43.076 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:44.078 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:45.049 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:45.050 7f8fa80fd720 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:45.050 7f8fa80fd720 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:45.051 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:45.051 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:45.051 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:45.051 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:45.051 7f8fa80fd720 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:45.051 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:45.051 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:45.051 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:45.051 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:45.051 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:45.051 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:45.080 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:46.082 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:47.083 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:48.086 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:49.053 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:49.055 7f8fa80fd720 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:49.055 7f8fa80fd720 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:49.055 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:49.056 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:49.056 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:49.056 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:49.057 7f8fa80fd720 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:49.057 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:49.058 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:49.058 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:49.058 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:49.058 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:49.059 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:49.087 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:50.089 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:51.090 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:52.092 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:53.056 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:53.058 7f8fa80fd720 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:53.058 7f8fa80fd720 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:53.058 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:53.059 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:53.059 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:53.059 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:53.060 7f8fa80fd720 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:53.061 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:53.061 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:53.061 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:53.061 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:53.061 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:53.062 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:53.093 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:54.095 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:55.097 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:56.098 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:57.060 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:57.062 7f8fa80fd720 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:57.062 7f8fa80fd720 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:57.063 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:57.063 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:57.063 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:57.064 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:57.064 7f8fa80fd720 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:57.065 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:57.065 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:57.066 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:57.066 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:57.067 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:57.067 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:57.099 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:58.101 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:59.102 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:00.104 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:01.063 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:01.064 7f8fa80fd720 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:01.064 7f8fa80fd720 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:01.064 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:01.065 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:01.065 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:01.065 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:01.065 7f8fa80fd720 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:01.065 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:01.065 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:01.065 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:01.065 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:01.065 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:01.065 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:01.106 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:02.107 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:03.108 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:04.110 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:05.065 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:05.067 7f8fa80fd720 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:05.068 7f8fa80fd720 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:05.068 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:05.068 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:05.068 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:05.069 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:05.069 7f8fa80fd720 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:05.070 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:05.070 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:05.070 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:05.070 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:05.071 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:05.071 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:05.111 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:06.114 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:07.116 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:08.117 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:09.069 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:09.071 7f8fa80fd720 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:09.071 7f8fa80fd720 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:09.071 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:09.072 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:09.072 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:09.072 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:09.073 7f8fa80fd720 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:09.073 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:09.074 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:09.074 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:09.074 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:09.074 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:09.075 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:09.118 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:10.120 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:11.122 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:12.124 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:13.073 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:13.075 7f8fa80fd720 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:13.075 7f8fa80fd720 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:13.075 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:13.075 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:13.075 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:13.075 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:13.076 7f8fa80fd720 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:13.076 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:13.076 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:13.076 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:13.076 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:13.076 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:13.076 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:13.126 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:14.127 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:15.128 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:16.130 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:17.076 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:17.077 7f8fa80fd720 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:17.077 7f8fa80fd720 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:17.077 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:17.077 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:17.077 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:17.077 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:17.077 7f8fa80fd720 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:17.077 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:17.078 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:17.078 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:17.078 7f8fa80fd720 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:17.078 7f8fa80fd720 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:17.078 7f8fa80fd720 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:17.132 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:18.134 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:19.136 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:20.138 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:21.079 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:21.080 7f8fa99a3860 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:21.081 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:21.081 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:21.081 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:21.082 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:21.082 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:21.083 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:21.083 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:21.083 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:21.083 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:21.084 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:21.084 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:21.084 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:21.140 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:22.141 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:23.143 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:24.144 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:25.082 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:25.084 7f8fa99a3860 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:25.085 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:25.085 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:25.085 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:25.085 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:25.086 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:25.086 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:25.087 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:25.087 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:25.087 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:25.087 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:25.087 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:25.088 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:25.146 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:26.148 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:27.150 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:28.152 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:29.085 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:29.086 7f8fa99a3860 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:29.086 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:29.086 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:29.086 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:29.087 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:29.087 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:29.087 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:29.087 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:29.087 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:29.087 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:29.087 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:29.087 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:29.087 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:29.153 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:30.155 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:31.156 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:32.158 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:33.088 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:33.090 7f8fa99a3860 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:33.090 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:33.090 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:33.091 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:33.091 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:33.091 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:33.092 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:33.092 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:33.093 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:33.093 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:33.093 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:33.094 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:33.094 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:33.159 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:34.162 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:35.163 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:36.165 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:37.091 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:37.093 7f8fa99a3860 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:37.094 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:37.094 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:37.094 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:37.094 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:37.095 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:37.096 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:37.096 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:37.096 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:37.096 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:37.097 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:37.097 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:37.097 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:37.167 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:38.168 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:39.170 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:40.171 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:41.094 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:41.096 7f8fa99a3860 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:41.096 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:41.096 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:41.097 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:41.097 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:41.097 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:41.098 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:41.098 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:41.098 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:41.099 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:41.099 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:41.099 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:41.099 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:41.173 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:42.175 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:43.176 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:44.178 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:45.097 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:45.099 7f8fa99a3860 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:45.099 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:45.099 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:45.100 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:45.100 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:45.100 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:45.100 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:45.100 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:45.100 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:45.100 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:45.100 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:45.100 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:45.101 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:45.179 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:46.181 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:47.182 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:48.184 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:49.100 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:49.101 7f8fa99a3860 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:49.101 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:49.101 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:49.102 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:49.102 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:49.102 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:49.102 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:49.102 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:49.102 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:49.102 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:49.102 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:49.102 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:49.103 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:49.186 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:50.187 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:51.189 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:52.191 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:53.103 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:53.105 7f8fa99a3860 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:53.106 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:53.106 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:53.107 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:53.107 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:53.108 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:53.109 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:53.109 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:53.110 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:53.110 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:53.110 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:53.111 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:53.111 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:53.193 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:54.194 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:55.196 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:56.198 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:56.953 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-3993-1'], 'obj_dict': {'bgp_router_parameters': {'address': '10.20.0.14', 'address_families': {'family': ['route-target', 'inet-vpn', 'e-vpn', 'erm-vpn', 'inet6-vpn']}, 'admin_down': False, 'auth_data': None, 'autonomous_system': 64512, 'cluster_id': None, 'gateway_address': None, 'hold_time': 0, 'identifier': '10.20.0.14', 'ipv6_gateway_address': None, 'local_autonomous_system': None, 'port': 179, 'router_type': 'control-node', 'source_port': None, 'vendor': 'contrail'}, 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3993-1', 'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-3993-1'], 'id_perms': {'created': '2025-09-01T01:58:56.941903', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:58:56.941903', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10796065273562621136, 'uuid_mslong': 17667355873221429929}}, 'parent_type': 'routing-instance', 'parent_uuid': '8105519c-daec-49ab-a7a7-a352adaaccd7', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'f52f0e8f-9fc6-4aa9-95d3-542a92a164d0'}, 'oper': 'CREATE', 'request-id': 'req-fbe82a3d-e0cd-4f3c-a5ca-f9701286b080', 'type': 'bgp_router', 'uuid': 'f52f0e8f-9fc6-4aa9-95d3-542a92a164d0'} 09/01/2025 01:58:57.106 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:57.107 7f8fa99a3860 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:57.107 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:57.107 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:57.107 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:57.107 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:57.107 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:57.108 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:57.108 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:57.108 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:57.108 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:57.108 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:57.108 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:57.108 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:57.199 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:57.218 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-0e8421c4-a285-47b4-9497-f3a607824657', 'type': 'global_system_config', 'uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500'} 09/01/2025 01:58:58.203 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:59.204 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:00.205 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:01.108 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:01.110 7f8fa99a3860 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:01.111 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:01.111 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:01.111 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:01.112 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:01.112 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:01.113 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:01.113 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:01.114 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:01.114 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:01.114 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:01.114 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:01.115 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:01.206 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:02.208 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:03.210 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:04.212 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:05.112 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:05.113 7f8fa99a3860 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:05.114 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:05.114 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:05.114 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:05.114 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:05.114 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:05.115 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:05.115 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:05.115 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:05.115 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:05.116 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:05.116 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:05.116 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:05.214 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:06.216 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:07.218 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:08.220 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:09.115 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:09.117 7f8fa99a3860 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:09.117 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:09.117 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:09.118 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:09.118 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:09.118 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:09.119 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:09.119 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:09.119 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:09.119 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:09.120 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:09.120 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:09.120 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:09.222 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:10.223 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:11.224 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:12.227 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:13.119 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:13.120 7f8fa99a3860 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:13.120 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:13.120 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:13.120 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:13.120 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:13.120 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:13.121 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:13.121 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:13.121 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:13.121 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:13.121 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:13.121 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:13.121 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:13.229 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:14.231 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:15.232 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:16.234 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:17.120 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:17.122 7f8fa99a3860 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:17.123 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:17.123 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:17.123 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:17.124 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:17.124 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:17.125 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:17.125 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:17.125 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:17.126 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:17.126 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:17.126 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:17.126 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:17.236 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:18.237 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:19.239 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:20.241 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:21.124 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:21.126 7f8fa99a3860 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:21.127 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:21.127 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:21.127 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:21.127 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:21.128 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:21.128 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:21.129 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:21.129 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:21.129 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:21.129 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:21.130 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:21.130 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:21.242 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:22.244 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:23.246 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:24.248 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:25.126 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:25.128 7f8fa99a3860 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:25.128 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:25.128 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:25.128 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:25.128 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:25.128 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:25.128 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:25.128 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:25.128 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:25.128 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:25.129 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:25.129 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:25.129 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:25.249 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:26.250 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:27.252 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:28.253 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:29.129 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:29.132 7f8fa99a3860 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:29.132 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:29.132 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:29.133 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:29.133 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:29.133 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:29.134 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:29.134 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:29.135 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:29.135 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:29.135 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:29.135 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:29.135 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:29.255 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:30.256 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:31.258 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:32.260 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:33.133 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:33.135 7f8fa99a3860 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:33.136 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:33.136 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:33.137 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:33.137 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:33.137 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:33.138 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:33.138 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:33.138 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:33.139 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:33.139 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:33.139 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:33.139 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:33.262 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:34.264 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:35.266 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:36.267 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:37.137 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:37.138 7f8fa99a3860 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:37.138 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:37.138 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:37.138 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:37.138 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:37.138 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:37.138 7f8fa99a3860 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:37.139 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:37.139 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:37.139 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:37.139 7f8fa99a3860 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:37.139 7f8fa99a3860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:37.139 7f8fa99a3860 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:37.276 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:38.278 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:39.235 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3993-1.'], 'obj_dict': {'database_node_ip_address': '10.0.0.56', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3993-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3993-1.'], 'id_perms': {'created': '2025-09-01T01:59:39.230073', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:59:39.230073', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11940803800317669233, 'uuid_mslong': 12965038512893676413}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'b3ed11d5-f449-4b7d-a5b6-41ca9291af71'}, 'oper': 'CREATE', 'request-id': 'req-f6b597f5-acb0-41b3-ae83-48a287481d21', 'type': 'database_node', 'uuid': 'b3ed11d5-f449-4b7d-a5b6-41ca9291af71'} 09/01/2025 01:59:39.279 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:40.280 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:41.139 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:41.140 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:41.141 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:41.141 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:41.141 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:41.141 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:41.141 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:41.141 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:41.141 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:41.141 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:41.141 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:41.141 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:41.142 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:41.142 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:41.281 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:42.283 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:43.284 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:44.286 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:45.141 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:45.144 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:45.144 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:45.145 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:45.145 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:45.145 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:45.145 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:45.146 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:45.147 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:45.147 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:45.147 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:45.148 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:45.148 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:45.148 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:45.287 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:46.288 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:47.290 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:48.292 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:49.144 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:49.147 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:49.147 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:49.148 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:49.148 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:49.149 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:49.149 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:49.150 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:49.150 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:49.151 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:49.151 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:49.151 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:49.151 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:49.152 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:49.293 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:50.295 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:51.297 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:52.298 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:53.148 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:53.150 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:53.150 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:53.150 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:53.150 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:53.151 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:53.151 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:53.151 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:53.152 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:53.152 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:53.152 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:53.152 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:53.153 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:53.153 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:53.300 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:54.302 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:55.304 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:56.307 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:57.151 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:57.152 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:57.152 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:57.152 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:57.152 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:57.153 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:57.153 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:57.153 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:57.153 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:57.153 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:57.153 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:57.153 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:57.153 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:57.153 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:57.309 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:58.311 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:59.313 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:00.315 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:01.154 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:01.155 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:01.155 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:01.155 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:01.155 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:01.156 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:01.156 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:01.156 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:01.156 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:01.156 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:01.156 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:01.156 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:01.156 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:01.156 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:01.317 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:02.319 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:03.320 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:04.322 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:05.157 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:05.160 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:05.160 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:05.161 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:05.161 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:05.162 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:05.162 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:05.163 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:05.164 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:05.164 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:05.165 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:05.165 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:05.165 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:05.166 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:05.323 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:06.325 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:07.327 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:08.328 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:09.161 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:09.163 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:09.163 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:09.163 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:09.164 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:09.164 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:09.164 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:09.165 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:09.165 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:09.166 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:09.166 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:09.166 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:09.166 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:09.166 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:09.330 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:10.332 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:11.334 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:12.335 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:13.164 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:13.166 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:13.167 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:13.167 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:13.167 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:13.167 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:13.168 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:13.169 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:13.169 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:13.169 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:13.169 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:13.170 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:13.170 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:13.170 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:13.336 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:14.338 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:15.339 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:16.341 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:17.167 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:17.168 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:17.168 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:17.168 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:17.168 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:17.168 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:17.168 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:17.169 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:17.169 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:17.169 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:17.169 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:17.169 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:17.169 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:17.169 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:17.343 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:18.344 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:19.346 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:20.348 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:21.169 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:21.172 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:21.172 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:21.172 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:21.173 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:21.173 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:21.173 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:21.174 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:21.174 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:21.175 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:21.175 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:21.175 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:21.175 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:21.176 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:21.349 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:22.351 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:23.352 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:24.354 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:25.173 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:25.175 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:25.176 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:25.176 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:25.177 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:25.177 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:25.177 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:25.177 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:25.178 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:25.179 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:25.179 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:25.179 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:25.179 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:25.179 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:25.360 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:26.361 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:27.243 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3993-1.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.56', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3993-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3993-1.'], 'id_perms': {'created': '2025-09-01T02:00:27.235470', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T02:00:27.235470', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12323209430845218635, 'uuid_mslong': 3889598271637308046}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '35faa105-23ea-428e-ab04-d5afaa45df4b'}, 'oper': 'CREATE', 'request-id': 'req-39a1b43c-ee87-49bf-b77f-cc837784a2a9', 'type': 'analytics_node', 'uuid': '35faa105-23ea-428e-ab04-d5afaa45df4b'} 09/01/2025 02:00:27.363 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:28.364 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:29.175 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:29.178 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:29.178 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:29.179 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:29.179 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:29.180 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:29.180 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:29.181 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:29.182 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:29.182 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:29.182 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:29.183 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:29.183 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:29.183 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:29.366 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:30.368 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:31.370 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:32.371 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:33.179 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:33.182 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:33.182 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:33.182 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:33.183 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:33.183 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:33.183 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:33.184 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:33.184 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:33.185 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:33.185 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:33.185 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:33.185 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:33.186 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:33.373 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:34.375 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:35.376 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:36.377 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:37.183 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:37.184 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:37.184 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:37.184 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:37.184 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:37.184 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:37.185 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:37.185 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:37.185 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:37.185 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:37.185 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:37.185 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:37.185 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:37.185 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:37.379 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:38.381 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:39.382 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:40.383 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:41.185 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:41.187 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:41.188 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:41.188 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:41.189 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:41.189 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:41.189 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:41.190 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:41.190 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:41.191 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:41.191 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:41.191 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:41.192 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:41.192 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:41.385 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:42.386 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:43.387 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:44.389 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:45.188 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:45.189 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:45.190 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:45.190 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:45.190 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:45.190 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:45.190 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:45.190 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:45.190 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:45.191 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:45.191 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:45.191 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:45.191 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:45.191 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:45.391 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:46.392 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:47.394 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:48.396 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:48.725 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3993-1.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.56', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3993-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3993-1.'], 'id_perms': {'created': '2025-09-01T02:00:48.714306', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T02:00:48.714306', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13408469371240238503, 'uuid_mslong': 10676906133802273316}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '942bfd8f-5d79-4a24-ba14-73da1e7acda7'}, 'oper': 'CREATE', 'request-id': 'req-d9fddf6e-4f72-4e37-b794-27c0e82736cc', 'type': 'analytics_alarm_node', 'uuid': '942bfd8f-5d79-4a24-ba14-73da1e7acda7'} 09/01/2025 02:00:49.191 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:49.192 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:49.193 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:49.193 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:49.193 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:49.193 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:49.193 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:49.193 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:49.194 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:49.194 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:49.194 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:49.194 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:49.194 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:49.194 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:49.398 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:50.399 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:51.401 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:52.403 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:53.193 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:53.195 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:53.195 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:53.195 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:53.196 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:53.196 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:53.196 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:53.197 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:53.197 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:53.198 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:53.198 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:53.198 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:53.199 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:53.199 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:53.405 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:54.407 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:55.409 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:56.410 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:57.196 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:57.199 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:57.199 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:57.199 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:57.200 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:57.200 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:57.200 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:57.201 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:57.201 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:57.201 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:57.202 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:57.202 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:57.202 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:57.202 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:57.412 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:58.414 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:59.416 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:00.416 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:01.199 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:01.201 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:01.202 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:01.202 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:01.202 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:01.203 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:01.203 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:01.203 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:01.204 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:01.204 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:01.204 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:01.204 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:01.205 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:01.205 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:01.417 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:02.420 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:03.422 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:04.424 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:05.004 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3993-1.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.56', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3993-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3993-1.'], 'id_perms': {'created': '2025-09-01T02:01:04.990043', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T02:01:04.990043', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11043142725590816080, 'uuid_mslong': 5586765827790422298}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '4d882deb-a0bd-411a-9941-1fccc2c79550'}, 'oper': 'CREATE', 'request-id': 'req-05faaaa3-e349-456f-a99e-6905adfb8346', 'type': 'analytics_snmp_node', 'uuid': '4d882deb-a0bd-411a-9941-1fccc2c79550'} 09/01/2025 02:01:05.202 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:05.204 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:05.205 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:05.205 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:05.205 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:05.206 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:05.206 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:05.207 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:05.207 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:05.207 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:05.207 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:05.208 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:05.208 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:05.208 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:05.425 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:06.427 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:07.428 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:08.430 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:09.206 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:09.208 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:09.208 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:09.209 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:09.209 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:09.209 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:09.209 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:09.210 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:09.210 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:09.211 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:09.211 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:09.211 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:09.211 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:09.211 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:09.432 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:10.434 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:11.435 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:12.436 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:13.209 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:13.210 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:13.210 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:13.210 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:13.210 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:13.211 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:13.211 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:13.211 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:13.211 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:13.211 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:13.211 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:13.211 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:13.211 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:13.211 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:13.438 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:14.439 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:15.441 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:16.443 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:17.212 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:17.214 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:17.214 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:17.214 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:17.215 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:17.215 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:17.215 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:17.216 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:17.216 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:17.217 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:17.217 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:17.217 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:17.217 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:17.217 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:17.444 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:18.446 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:19.448 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:20.449 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:21.215 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:21.217 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:21.217 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:21.218 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:21.218 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:21.218 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:21.218 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:21.219 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:21.219 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:21.220 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:21.220 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:21.220 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:21.220 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:21.220 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:21.451 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:22.453 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:23.454 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:24.456 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:25.218 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:25.220 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:25.221 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:25.221 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:25.222 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:25.222 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:25.222 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:25.223 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:25.223 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:25.223 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:25.223 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:25.224 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:25.224 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:25.224 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:25.458 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:26.460 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:27.462 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:28.464 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:29.221 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:29.223 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:29.224 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:29.224 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:29.224 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:29.225 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:29.225 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:29.226 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:29.226 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:29.226 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:29.227 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:29.227 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:29.227 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:29.227 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:29.466 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:30.467 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:31.468 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:32.469 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:33.224 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:33.227 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:33.228 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:33.228 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:33.229 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:33.229 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:33.229 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:33.230 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:33.231 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:33.231 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:33.232 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:33.232 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:33.233 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:33.233 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:33.471 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:34.472 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:35.474 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:36.476 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:37.228 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:37.230 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:37.231 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:37.231 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:37.231 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:37.231 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:37.232 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:37.232 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:37.233 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:37.233 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:37.233 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:37.233 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:37.234 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:37.234 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:37.477 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:38.478 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:39.480 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:40.482 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:41.231 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:41.232 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:41.232 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:41.232 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:41.232 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:41.232 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:41.232 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:41.233 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:41.233 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:41.233 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:41.233 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:41.233 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:41.233 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:41.233 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:41.484 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:42.485 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:43.487 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:44.488 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:45.233 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:45.235 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:45.235 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:45.235 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:45.235 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:45.236 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:45.236 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:45.236 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:45.236 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:45.237 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:45.237 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:45.237 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:45.237 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:45.237 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:45.489 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:46.491 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:47.492 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:48.494 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:49.236 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:49.239 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:49.239 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:49.239 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:49.240 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:49.240 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:49.240 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:49.241 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:49.241 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:49.241 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:49.242 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:49.242 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:49.242 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:49.242 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:49.496 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:50.499 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:51.500 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:52.502 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:53.240 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:53.242 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:53.242 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:53.242 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:53.243 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:53.243 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:53.243 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:53.244 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:53.245 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:53.245 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:53.245 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:53.245 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:53.246 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:53.246 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:53.504 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:54.505 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:55.507 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:56.509 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:57.243 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:57.245 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:57.245 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:57.246 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:57.246 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:57.246 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:57.246 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:57.247 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:57.247 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:57.248 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:57.248 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:57.248 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:57.249 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:57.250 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:57.511 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:58.512 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:59.514 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:00.516 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:01.246 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:01.249 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:01.249 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:01.250 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:01.250 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:01.251 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:01.251 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:01.252 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:01.252 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:01.253 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:01.253 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:01.253 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:01.253 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:01.254 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:01.519 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:02.520 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:03.521 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:04.523 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:05.249 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:05.252 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:05.252 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:05.252 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:05.253 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:05.253 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:05.253 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:05.254 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:05.254 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:05.255 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:05.255 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:05.255 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:05.256 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:05.256 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:05.525 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:06.527 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:07.528 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:08.530 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:09.253 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:09.255 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:09.255 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:09.255 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:09.256 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:09.256 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:09.256 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:09.257 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:09.257 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:09.258 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:09.258 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:09.258 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:09.258 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:09.258 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:09.532 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:10.534 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:11.535 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:12.537 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:13.257 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:13.259 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:13.259 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:13.259 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:13.260 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:13.260 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:13.260 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:13.261 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:13.261 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:13.261 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:13.261 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:13.262 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:13.262 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:13.262 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:13.539 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:14.540 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:15.542 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:16.544 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:17.259 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:17.260 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:17.260 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:17.260 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:17.261 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:17.261 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:17.261 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:17.261 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:17.261 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:17.261 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:17.261 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:17.261 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:17.262 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:17.262 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:17.546 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:18.548 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:19.549 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:20.550 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:21.261 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:21.263 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:21.264 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:21.264 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:21.264 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:21.265 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:21.265 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:21.265 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:21.266 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:21.266 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:21.266 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:21.266 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:21.267 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:21.267 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:21.552 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:22.554 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:23.555 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:24.557 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:25.265 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:25.267 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:25.268 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:25.268 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:25.268 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:25.269 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:25.269 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:25.269 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:25.270 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:25.270 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:25.270 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:25.270 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:25.271 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:25.271 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:25.559 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:26.560 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:27.563 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:28.564 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:29.268 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:29.270 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:29.271 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:29.271 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:29.271 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:29.272 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:29.272 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:29.273 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:29.273 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:29.273 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:29.274 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:29.274 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:29.274 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:29.274 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:29.566 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:30.568 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:31.570 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:32.572 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:33.271 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:33.275 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:33.276 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:33.276 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:33.277 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:33.277 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:33.277 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:33.278 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:33.278 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:33.278 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:33.279 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:33.279 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:33.279 7f8fa80fd0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:33.279 7f8fa80fd0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:33.574 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:34.576 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:35.577 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:36.578 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:37.276 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:37.279 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:37.279 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:37.280 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:37.281 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:37.281 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:37.281 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:37.282 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:37.283 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:37.283 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:37.284 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:37.284 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:37.284 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:37.285 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:37.580 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:38.582 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:39.583 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:40.585 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:41.280 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:41.281 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:41.282 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:41.282 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:41.282 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:41.282 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:41.282 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:41.283 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:41.283 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:41.283 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:41.283 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:41.284 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:41.284 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:41.284 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:41.587 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:42.588 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:43.590 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:44.591 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:45.282 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:45.285 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:45.285 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:45.286 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:45.286 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:45.286 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:45.287 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:45.287 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:45.288 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:45.288 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:45.288 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:45.289 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:45.289 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:45.289 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:45.593 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:46.595 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:47.597 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:48.599 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:49.286 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:49.288 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:49.289 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:49.289 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:49.289 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:49.289 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:49.289 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:49.289 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:49.290 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:49.290 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:49.290 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:49.290 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:49.290 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:49.290 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:49.600 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:50.602 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:51.604 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:52.606 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:53.289 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:53.292 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:53.292 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:53.292 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:53.293 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:53.293 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:53.293 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:53.294 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:53.294 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:53.295 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:53.295 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:53.295 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:53.295 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:53.296 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:53.608 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:54.609 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:55.610 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:56.612 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:57.292 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:57.294 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:57.295 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:57.295 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:57.296 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:57.296 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:57.296 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:57.297 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:57.298 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:57.298 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:57.298 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:57.298 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:57.299 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:57.299 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:57.614 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:58.616 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:59.617 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:00.620 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:01.296 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:01.298 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:01.299 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:01.299 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:01.299 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:01.300 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:01.300 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:01.301 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:01.301 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:01.301 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:01.302 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:01.302 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:01.302 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:01.302 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:01.621 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:02.622 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:03.624 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:04.626 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:05.300 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:05.302 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:05.302 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:05.302 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:05.303 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:05.303 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:05.303 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:05.304 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:05.304 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:05.304 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:05.304 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:05.305 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:05.305 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:05.305 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:05.628 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:06.630 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:07.631 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:08.632 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:09.303 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:09.305 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:09.305 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:09.305 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:09.306 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:09.306 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:09.306 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:09.307 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:09.307 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:09.307 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:09.307 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:09.307 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:09.308 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:09.308 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:09.634 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:10.635 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:11.637 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:12.639 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:13.305 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:13.307 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:13.308 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:13.308 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:13.308 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:13.309 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:13.309 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:13.310 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:13.310 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:13.310 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:13.310 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:13.311 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:13.311 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:13.311 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:13.641 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:14.642 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:15.644 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:16.646 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:17.308 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:17.310 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:17.310 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:17.310 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:17.310 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:17.310 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:17.310 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:17.311 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:17.311 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:17.311 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:17.311 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:17.311 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:17.311 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:17.311 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:17.648 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:18.650 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:19.652 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:20.654 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:21.311 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:21.313 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:21.314 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:21.314 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:21.314 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:21.314 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:21.314 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:21.315 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:21.315 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:21.316 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:21.316 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:21.316 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:21.316 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:21.316 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:21.656 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:22.658 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:23.661 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:24.663 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:25.314 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:25.317 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:25.318 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:25.318 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:25.319 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:25.319 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:25.319 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:25.320 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:25.321 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:25.321 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:25.322 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:25.322 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:25.322 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:25.323 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:25.665 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:26.667 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:27.669 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:28.671 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:29.318 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:29.320 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:29.321 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:29.321 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:29.321 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:29.321 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:29.322 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:29.322 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:29.323 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:29.323 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:29.323 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:29.323 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:29.324 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:29.324 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:29.672 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:30.674 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:31.676 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:32.678 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:33.321 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:33.323 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:33.324 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:33.324 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:33.324 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:33.325 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:33.325 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:33.326 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:33.326 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:33.326 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:33.326 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:33.327 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:33.327 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:33.327 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:33.679 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:34.681 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:35.683 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:36.685 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:37.325 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:37.327 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:37.327 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:37.327 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:37.328 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:37.328 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:37.328 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:37.329 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:37.329 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:37.329 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:37.330 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:37.330 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:37.330 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:37.330 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:37.687 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:38.689 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:39.691 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:40.692 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:41.327 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:41.329 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:41.330 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:41.330 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:41.331 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:41.331 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:41.331 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:41.332 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:41.332 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:41.332 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:41.333 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:41.333 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:41.333 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:41.333 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:41.694 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:42.696 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:43.698 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:44.699 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:45.331 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:45.333 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:45.333 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:45.333 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:45.333 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:45.334 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:45.334 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:45.334 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:45.334 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:45.335 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:45.335 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:45.335 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:45.335 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:45.335 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:45.701 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:46.702 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:47.704 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:48.706 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:49.334 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:49.335 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:49.335 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:49.336 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:49.336 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:49.336 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:49.336 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:49.336 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:49.336 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:49.337 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:49.337 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:49.337 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:49.337 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:49.337 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:49.708 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:50.709 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:51.711 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:52.713 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:53.337 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:53.339 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:53.339 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:53.339 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:53.340 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:53.340 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:53.341 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:53.341 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:53.342 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:53.342 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:53.343 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:53.343 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:53.343 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:53.343 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:53.714 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:54.716 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:55.717 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:56.206 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-5d92ffbe-d671-4442-9a6a-cbeb361f2ec9', 'type': 'global_system_config', 'uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500'} 09/01/2025 02:03:56.348 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3993-1'], 'obj_dict': {'display_name': 'cn-jenkins-deploy-platform-ansible-os-3993-1', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3993-1'], 'id_perms': {'created': '2025-09-01T02:03:56.337011', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T02:03:56.337011', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13614653776856471694, 'uuid_mslong': 13705629618940693837}}, 'parent_type': 'global-system-config', 'parent_uuid': '7eedf3ea-84b7-4623-96cf-7fecf55ca500', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'be342d85-dc08-4d4d-bcf0-f779fd75d88e', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.14'}, 'oper': 'CREATE', 'request-id': 'req-d72e219f-e9c4-4d0e-b3c6-3076bbe59a13', 'type': 'virtual_router', 'uuid': 'be342d85-dc08-4d4d-bcf0-f779fd75d88e'} 09/01/2025 02:03:56.718 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:56.734 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3993-1', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3993-1', 'vhost0'], 'id_perms': {'created': '2025-09-01T02:03:56.715607', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T02:03:56.715607', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10136671416836121550, 'uuid_mslong': 1509424230754177005}}, 'parent_type': 'virtual-router', 'parent_uuid': 'be342d85-dc08-4d4d-bcf0-f779fd75d88e', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '14f28d57-3178-4fed-8cac-b0f27d5f63ce', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:14:f2:8d:57:31']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': 'c39bc5a7-3ad4-482b-9be0-1adbb0ac1242'}]}, 'oper': 'CREATE', 'request-id': 'req-20622f07-cda6-43e3-8080-4a5fb623365b', 'type': 'virtual_machine_interface', 'uuid': '14f28d57-3178-4fed-8cac-b0f27d5f63ce'} 09/01/2025 02:03:56.772 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3993-1', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-20622f07-cda6-43e3-8080-4a5fb623365b', 'type': 'virtual_machine_interface', 'uuid': '14f28d57-3178-4fed-8cac-b0f27d5f63ce'} 09/01/2025 02:03:57.307 7f8fa80c59a0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-375f7284-d086-48a3-96ae-3f89b820eabc', 'type': 'access_control_list', 'uuid': '699e2991-a098-40cb-81fd-a7e669d68ed7'} 09/01/2025 02:03:57.340 7f8fa80c5e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:57.341 7f8fa80c5e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:57.341 7f8fa80c5e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:57.341 7f8fa80c5e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:57.341 7f8fa80c5e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:57.341 7f8fa80c5e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:57.341 7f8fa80c5e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:57.342 7f8fa80c5e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:57.342 7f8fa80c5e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:57.342 7f8fa80c5e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:57.342 7f8fa80c5e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:57.342 7f8fa80c5e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:57.342 7f8fa80c5e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:57.343 7f8fa80c5e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:57.720 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:58.722 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:59.724 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:00.726 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:01.342 7f8fa80c5e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:01.344 7f8fa80c5e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:01.344 7f8fa80c5e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:01.344 7f8fa80c5e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:01.344 7f8fa80c5e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:01.345 7f8fa80c5e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:01.345 7f8fa80c5e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:01.345 7f8fa80c5e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:01.346 7f8fa80c5e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:01.346 7f8fa80c5e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:01.346 7f8fa80c5e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:01.346 7f8fa80c5e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:01.347 7f8fa80c5e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:01.347 7f8fa80c5e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:01.729 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:02.730 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:03.732 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:04.732 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:05.344 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:05.346 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:05.346 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:05.346 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:05.346 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:05.346 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:05.346 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:05.346 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:05.346 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:05.347 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:05.347 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:05.347 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:05.347 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:05.347 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:05.734 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:06.736 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:07.737 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:07.831 7f8fa80c5f40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:04:08.740 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:09.346 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:09.349 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:09.349 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:09.349 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:09.350 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:09.350 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:09.350 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:09.351 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:09.351 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:09.351 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:09.351 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:09.351 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:09.352 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:09.352 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:09.742 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:10.744 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:11.746 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:12.747 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:13.350 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:13.352 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:13.352 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:13.353 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:13.353 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:13.353 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:13.354 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:13.355 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:13.355 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:13.355 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:13.355 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:13.356 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:13.359 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:13.359 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:13.749 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:14.751 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:15.753 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:16.755 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:17.353 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:17.355 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:17.356 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:17.356 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:17.356 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:17.356 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:17.357 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:17.357 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:17.358 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:17.358 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:17.358 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:17.358 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:17.359 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:17.359 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:17.757 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:18.759 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:19.761 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:20.740 7f8fa80c5f40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:04:20.763 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:21.356 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:21.359 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:21.359 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:21.359 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:21.360 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:21.360 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:21.360 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:21.362 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:21.362 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:21.363 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:21.363 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:21.363 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:21.363 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:21.363 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:21.764 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:22.766 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:23.768 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:24.771 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:25.359 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:25.361 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:25.362 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:25.362 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:25.363 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:25.363 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:25.363 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:25.364 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:25.365 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:25.365 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:25.365 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:25.365 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:25.366 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:25.366 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:25.772 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:26.774 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:27.776 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:28.778 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:29.363 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:29.366 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:29.367 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:29.367 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:29.367 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:29.368 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:29.368 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:29.369 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:29.369 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:29.370 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:29.370 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:29.370 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:29.371 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:29.371 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:29.780 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:30.782 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:31.783 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:32.785 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:33.367 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:33.369 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:33.370 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:33.370 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:33.370 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:33.370 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:33.371 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:33.371 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:33.372 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:33.372 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:33.372 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:33.372 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:33.373 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:33.373 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:33.653 7f8fa80c5f40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:04:33.787 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:34.788 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:35.790 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:36.792 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:37.370 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:37.373 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:37.373 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:37.373 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:37.374 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:37.374 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:37.374 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:37.375 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:37.375 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:37.376 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:37.376 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:37.376 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:37.376 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:37.377 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:37.794 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:38.796 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:39.798 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:40.798 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:41.374 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:41.376 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:41.377 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:41.377 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:41.377 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:41.378 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:41.378 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:41.379 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:41.379 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:41.379 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:41.380 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:41.380 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:41.380 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:41.381 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:41.800 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:42.801 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:43.803 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:44.804 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:45.376 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:45.378 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:45.378 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:45.378 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:45.378 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:45.379 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:45.379 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:45.379 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:45.379 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:45.379 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:45.379 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:45.380 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:45.380 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:45.380 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:45.807 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:46.808 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:46.935 7f8fa80c5f40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:04:47.810 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:48.812 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:49.379 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:49.381 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:49.382 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:49.382 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:49.382 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:49.383 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:49.383 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:49.384 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:49.384 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:49.384 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:49.385 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:49.385 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:49.385 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:49.385 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:49.813 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:50.815 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:51.816 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:52.819 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:53.383 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:53.385 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:53.385 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:53.385 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:53.386 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:53.386 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:53.386 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:53.387 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:53.387 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:53.387 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:53.387 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:53.387 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:53.388 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:53.388 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:53.821 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:54.822 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:55.824 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:56.826 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:57.385 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:57.387 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:57.387 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:57.388 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:57.388 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:57.388 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:57.389 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:57.389 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:57.390 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:57.390 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:57.390 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:57.390 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:57.391 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:57.391 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:57.828 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:58.831 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:59.649 7f8fa80c5f40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:04:59.833 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:00.835 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:01.389 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:01.391 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:01.391 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:01.391 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:01.392 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:01.392 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:01.392 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:01.393 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:01.393 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:01.394 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:01.394 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:01.394 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:01.394 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:01.394 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:01.836 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:02.838 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:03.839 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:04.841 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:05.392 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:05.394 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:05.394 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:05.394 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:05.395 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:05.395 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:05.395 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:05.396 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:05.396 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:05.396 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:05.396 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:05.396 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:05.397 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:05.397 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:05.843 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:06.844 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:07.846 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:08.848 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:09.394 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:09.396 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:09.396 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:09.396 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:09.397 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:09.397 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:09.397 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:09.398 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:09.398 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:09.398 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:09.398 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:09.398 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:09.399 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:09.399 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:09.850 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:10.851 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:11.853 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:12.855 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:12.930 7f8fa80c5f40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:05:13.397 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:13.400 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:13.400 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:13.401 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:13.401 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:13.401 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:13.401 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:13.402 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:13.402 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:13.403 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:13.403 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:13.404 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:13.404 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:13.404 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:13.857 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:14.859 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:15.862 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:16.864 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:17.401 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:17.403 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:17.404 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:17.404 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:17.404 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:17.405 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:17.405 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:17.406 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:17.406 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:17.406 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:17.406 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:17.406 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:17.407 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:17.407 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:17.866 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:18.867 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:19.869 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:20.871 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:21.404 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:21.407 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:21.407 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:21.408 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:21.408 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:21.409 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:21.409 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:21.410 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:21.410 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:21.411 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:21.411 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:21.411 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:21.412 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:21.412 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:21.873 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:22.875 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:23.876 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:24.880 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:25.407 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:25.408 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:25.409 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:25.409 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:25.409 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:25.409 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:25.409 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:25.409 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:25.409 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:25.410 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:25.410 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:25.410 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:25.410 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:25.410 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:25.883 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:26.021 7f8fa80c5f40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:05:26.885 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:27.887 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:28.890 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:29.409 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:29.411 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:29.411 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:29.411 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:29.412 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:29.412 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:29.412 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:29.413 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:29.413 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:29.413 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:29.413 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:29.414 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:29.414 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:29.414 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:29.892 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:30.894 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:31.896 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:32.898 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:33.412 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:33.414 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:33.414 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:33.414 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:33.415 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:33.415 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:33.415 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:33.416 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:33.417 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:33.417 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:33.417 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:33.417 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:33.418 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:33.418 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:33.900 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:34.902 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:35.904 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:36.906 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:37.415 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:37.417 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:37.418 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:37.418 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:37.418 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:37.419 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:37.419 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:37.420 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:37.420 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:37.420 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:37.420 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:37.421 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:37.421 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:37.421 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:37.908 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:38.867 7f8fa80c5f40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:05:38.909 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:39.910 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:40.912 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:41.418 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:41.419 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:41.420 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:41.420 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:41.420 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:41.420 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:41.420 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:41.420 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:41.420 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:41.420 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:41.421 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:41.421 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:41.421 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:41.421 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:41.914 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:42.916 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:43.918 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:44.920 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:45.420 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:45.422 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:45.423 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:45.423 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:45.423 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:45.424 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:45.424 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:45.425 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:45.425 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:45.425 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:45.426 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:45.426 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:45.426 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:45.426 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:45.921 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:46.923 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:47.926 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:48.927 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:49.424 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:49.426 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:49.426 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:49.426 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:49.427 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:49.427 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:49.427 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:49.428 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:49.428 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:49.429 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:49.429 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:49.429 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:49.429 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:49.430 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:49.929 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:50.931 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:51.686 7f8fa80c5f40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:05:51.933 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:52.935 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:53.427 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:53.428 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:53.428 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:53.428 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:53.429 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:53.429 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:53.429 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:53.429 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:53.429 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:53.429 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:53.429 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:53.429 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:53.430 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:53.430 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:53.936 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:54.938 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:55.940 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:56.941 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:57.429 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:57.432 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:57.432 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:57.433 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:57.433 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:57.433 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:57.433 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:57.434 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:57.434 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:57.435 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:57.435 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:57.435 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:57.435 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:57.435 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:57.943 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:58.946 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:59.948 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:00.950 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:01.433 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:01.436 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:01.436 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:01.437 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:01.437 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:01.437 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:01.438 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:01.439 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:01.439 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:01.439 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:01.439 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:01.440 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:01.440 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:01.440 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:01.951 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:02.952 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:03.954 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:04.505 7f8fa80c5f40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:06:04.956 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:05.437 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:05.438 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:05.438 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:05.438 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:05.438 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:05.439 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:05.439 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:05.439 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:05.439 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:05.439 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:05.439 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:05.439 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:05.439 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:05.439 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:05.959 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:06.960 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:07.962 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:08.963 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:09.439 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:09.441 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:09.442 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:09.442 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:09.442 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:09.442 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:09.443 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:09.443 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:09.444 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:09.444 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:09.444 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:09.445 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:09.445 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:09.445 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:09.965 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:10.968 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:11.970 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:12.972 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:13.443 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:13.445 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:13.445 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:13.445 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:13.446 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:13.446 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:13.446 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:13.447 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:13.447 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:13.448 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:13.448 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:13.448 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:13.448 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:13.449 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:13.975 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:14.977 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:15.978 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:16.980 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:17.446 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:17.448 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:17.448 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:17.449 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:17.449 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:17.449 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:17.450 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:17.450 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:17.451 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:17.451 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:17.451 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:17.451 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:17.452 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:17.452 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:17.971 7f8fa80c5f40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:06:17.982 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:18.983 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:19.984 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:20.986 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:21.449 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:21.451 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:21.452 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:21.452 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:21.452 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:21.453 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:21.453 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:21.454 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:21.454 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:21.454 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:21.455 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:21.455 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:21.455 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:21.455 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:21.988 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:22.989 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:23.992 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:24.994 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:25.452 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:25.453 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:25.454 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:25.454 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:25.454 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:25.454 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:25.454 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:25.455 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:25.455 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:25.455 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:25.455 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:25.455 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:25.455 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:25.455 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:25.996 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:26.997 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:27.999 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:29.001 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:29.455 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:29.457 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:29.458 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:29.458 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:29.459 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:29.459 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:29.459 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:29.460 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:29.460 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:29.460 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:29.461 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:29.461 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:29.461 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:29.461 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:30.003 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:30.800 7f8fa80c5f40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:06:31.004 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:32.007 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:33.009 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:33.458 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:33.460 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:33.461 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:33.461 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:33.461 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:33.461 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:33.462 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:33.462 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:33.462 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:33.463 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:33.463 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:33.463 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:33.463 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:33.464 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:34.011 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:35.013 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:36.015 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:37.017 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:37.461 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:37.462 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:37.463 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:37.463 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:37.463 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:37.463 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:37.463 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:37.464 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:37.464 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:37.464 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:37.464 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:37.464 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:37.464 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:37.464 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:38.019 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:39.020 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:40.021 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:41.022 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:41.463 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:41.465 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:41.466 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:41.466 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:41.466 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:41.466 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:41.467 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:41.467 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:41.467 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:41.468 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:41.468 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:41.468 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:41.468 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:41.468 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:42.024 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:43.026 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:43.623 7f8fa80c5f40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:06:44.028 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:45.030 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:45.467 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:45.469 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:45.469 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:45.470 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:45.470 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:45.471 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:45.471 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:45.472 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:45.472 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:45.473 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:45.473 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:45.473 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:45.473 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:45.474 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:46.031 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:47.033 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:48.034 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:49.036 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:49.470 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:49.472 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:49.472 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:49.473 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:49.473 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:49.473 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:49.474 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:49.474 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:49.474 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:49.475 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:49.475 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:49.475 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:49.475 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:49.476 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:50.038 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:51.040 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:52.041 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:53.042 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:53.473 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:53.474 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:53.474 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:53.474 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:53.474 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:53.474 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:53.474 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:53.474 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:53.474 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:53.475 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:53.475 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:53.475 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:53.475 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:53.475 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:54.044 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:55.046 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:56.047 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:56.509 7f8fa80c5f40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:06:57.050 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:57.475 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:57.477 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:57.477 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:57.478 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:57.478 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:57.478 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:57.479 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:57.480 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:57.480 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:57.481 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:57.481 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:57.482 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:57.482 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:57.483 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:58.052 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:59.054 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:00.056 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:01.058 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:01.477 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:01.479 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:01.479 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:01.479 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:01.480 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:01.480 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:01.480 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:01.481 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:01.481 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:01.481 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:01.481 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:01.481 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:01.481 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:01.482 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:02.060 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:03.062 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:04.064 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:05.065 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:05.481 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:05.483 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:05.483 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:05.483 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:05.484 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:05.484 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:05.485 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:05.486 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:05.486 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:05.487 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:05.487 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:05.487 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:05.487 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:05.487 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:06.067 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:07.069 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:08.071 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:09.072 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:09.336 7f8fa80c5f40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:07:09.484 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:09.486 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:09.486 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:09.486 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:09.487 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:09.487 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:09.487 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:09.488 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:09.489 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:09.489 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:09.489 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:09.490 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:09.490 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:09.490 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:10.074 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:11.076 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:12.079 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:13.081 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:13.486 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:13.488 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:13.488 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:13.488 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:13.488 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:13.489 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:13.489 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:13.489 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:13.489 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:13.490 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:13.490 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:13.490 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:13.490 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:13.490 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:14.083 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:15.085 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:16.087 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:17.089 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:17.489 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:17.491 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:17.491 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:17.492 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:17.492 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:17.492 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:17.493 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:17.493 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:17.494 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:17.494 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:17.494 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:17.494 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:17.495 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:17.495 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:18.092 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:19.095 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:20.096 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:21.098 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:21.491 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:21.494 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:21.494 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:21.494 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:21.495 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:21.495 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:21.495 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:21.496 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:21.496 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:21.497 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:21.497 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:21.497 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:21.498 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:21.498 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:22.100 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:22.654 7f8fa80c5f40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:07:23.102 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:24.104 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:25.107 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:25.495 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:25.498 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:25.498 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:25.498 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:25.499 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:25.499 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:25.499 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:25.500 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:25.500 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:25.501 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:25.501 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:25.501 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:25.502 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:25.502 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:26.109 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:27.111 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:28.112 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:29.114 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:29.498 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:29.499 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:29.500 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:29.500 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:29.500 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:29.500 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:29.500 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:29.501 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:29.501 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:29.501 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:29.501 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:29.501 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:29.502 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:29.502 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:30.116 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:31.118 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:32.119 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:33.120 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:33.501 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:33.503 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:33.503 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:33.504 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:33.504 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:33.504 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:33.505 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:33.505 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:33.506 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:33.506 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:33.506 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:33.507 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:33.507 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:33.507 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:34.122 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:35.124 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:35.672 7f8fa80c5f40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:07:36.126 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:37.128 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:37.504 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:37.506 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:37.507 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:37.507 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:37.507 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:37.507 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:37.508 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:37.509 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:37.509 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:37.509 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:37.509 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:37.510 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:37.510 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:37.510 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:38.130 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:39.132 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:40.134 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:41.136 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:41.508 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:41.510 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:41.510 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:41.510 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:41.511 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:41.511 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:41.511 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:41.512 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:41.512 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:41.512 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:41.513 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:41.513 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:41.513 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:41.513 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:42.138 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:43.141 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:44.143 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:45.145 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:45.510 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:45.512 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:45.513 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:45.513 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:45.513 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:45.514 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:45.514 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:45.515 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:45.515 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:45.515 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:45.515 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:45.516 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:45.516 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:45.516 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:46.146 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:47.148 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:48.150 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:48.691 7f8fa80c5f40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:07:49.152 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:49.514 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:49.516 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:49.517 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:49.517 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:49.518 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:49.518 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:49.518 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:49.519 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:49.520 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:49.520 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:49.520 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:49.521 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:49.521 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:49.521 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:50.155 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:51.156 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:52.158 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:53.159 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:53.518 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:53.520 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:53.520 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:53.520 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:53.521 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:53.521 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:53.521 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:53.522 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:53.523 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:53.523 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:53.523 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:53.523 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:53.524 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:53.524 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:54.161 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:55.163 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:56.165 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:57.166 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:57.520 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:57.522 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:57.522 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:57.522 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:57.523 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:57.523 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:57.523 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:57.524 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:57.524 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:57.524 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:57.524 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:57.524 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:57.525 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:57.525 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:58.168 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:59.170 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:00.171 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:01.174 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:01.523 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:01.524 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:01.524 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:01.524 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:01.524 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:01.524 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:01.524 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:01.525 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:01.525 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:01.525 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:01.525 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:01.525 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:01.525 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:01.525 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:01.564 7f8fa80c5f40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:08:02.175 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:03.176 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:04.179 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:05.181 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:05.525 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:05.527 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:05.528 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:05.528 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:05.529 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:05.529 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:05.529 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:05.531 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:05.531 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:05.531 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:05.532 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:05.532 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:05.533 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:05.533 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:06.183 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:07.186 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:08.188 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:09.190 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:09.528 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:09.529 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:09.529 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:09.529 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:09.529 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:09.530 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:09.530 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:09.530 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:09.530 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:09.530 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:09.530 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:09.531 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:09.531 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:09.531 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:10.192 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:11.194 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:12.196 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:13.199 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:13.530 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:13.531 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:13.531 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:13.531 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:13.531 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:13.531 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:13.531 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:13.532 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:13.532 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:13.532 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:13.532 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:13.532 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:13.532 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:13.532 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:14.200 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:14.496 7f8fa80c5f40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:08:15.202 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:16.204 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:17.206 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:17.532 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:17.534 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:17.535 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:17.535 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:17.535 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:17.536 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:17.536 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:17.537 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:17.537 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:17.537 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:17.537 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:17.538 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:17.538 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:17.538 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:18.208 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:19.211 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:20.213 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:21.214 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:21.535 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:21.536 7f8fa80c5f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:21.536 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:21.536 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:21.536 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:21.536 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:21.536 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:21.537 7f8fa80c5f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:21.537 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:21.537 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:21.537 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:21.537 7f8fa80c5f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:21.537 7f8fa80c5f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:21.537 7f8fa80c5f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:22.216 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:23.217 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:24.219 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:25.221 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:25.537 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:25.538 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:25.538 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:25.538 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:25.538 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:25.538 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:25.538 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:25.539 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:25.539 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:25.539 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:25.539 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:25.539 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:25.539 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:25.539 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:26.222 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:27.224 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:27.906 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:08:28.226 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:29.227 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:29.538 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:29.540 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:29.540 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:29.541 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:29.541 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:29.541 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:29.541 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:29.542 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:29.542 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:29.543 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:29.543 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:29.543 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:29.543 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:29.543 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:30.230 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:31.232 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:32.234 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:33.236 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:33.540 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:33.542 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:33.543 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:33.543 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:33.543 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:33.543 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:33.543 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:33.544 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:33.544 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:33.545 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:33.545 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:33.545 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:33.545 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:33.545 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:34.238 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:35.240 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:36.242 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:37.243 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:37.543 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:37.545 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:37.546 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:37.546 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:37.547 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:37.547 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:37.547 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:37.548 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:37.548 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:37.548 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:37.549 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:37.549 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:37.549 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:37.549 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:38.246 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:39.247 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:40.249 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:40.748 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:08:41.250 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:41.546 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:41.547 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:41.547 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:41.547 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:41.548 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:41.548 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:41.548 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:41.548 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:41.548 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:41.548 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:41.548 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:41.548 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:41.548 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:41.548 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:42.252 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:43.254 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:44.256 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:45.258 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:45.548 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:45.550 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:45.550 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:45.550 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:45.551 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:45.551 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:45.551 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:45.552 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:45.552 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:45.553 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:45.553 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:45.553 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:45.553 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:45.554 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:46.260 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:47.262 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:48.264 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:49.266 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:49.551 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:49.553 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:49.554 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:49.554 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:49.555 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:49.555 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:49.555 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:49.556 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:49.556 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:49.557 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:49.557 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:49.557 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:49.558 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:49.558 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:50.268 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:51.270 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:52.272 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:53.275 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:53.554 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:53.556 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:53.556 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:53.556 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:53.556 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:53.556 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:53.557 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:53.557 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:53.557 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:53.557 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:53.557 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:53.557 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:53.557 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:53.558 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:53.979 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:08:54.276 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:55.277 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:56.279 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:57.281 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:57.557 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:57.558 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:57.558 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:57.558 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:57.558 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:57.558 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:57.558 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:57.559 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:57.559 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:57.559 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:57.559 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:57.559 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:57.559 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:57.559 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:58.283 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:59.286 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:00.288 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:01.290 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:01.559 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:01.560 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:01.560 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:01.561 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:01.561 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:01.561 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:01.561 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:01.561 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:01.561 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:01.561 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:01.561 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:01.562 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:01.562 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:01.562 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:02.292 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:03.294 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:04.295 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:05.296 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:05.562 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:05.563 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:05.563 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:05.564 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:05.564 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:05.564 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:05.564 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:05.565 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:05.565 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:05.566 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:05.566 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:05.566 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:05.567 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:05.567 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:06.299 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:07.150 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:09:07.301 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:08.303 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:09.305 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:09.564 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:09.566 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:09.567 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:09.567 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:09.568 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:09.568 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:09.568 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:09.569 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:09.570 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:09.570 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:09.571 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:09.571 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:09.571 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:09.572 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:10.307 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:11.309 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:12.311 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:13.313 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:13.567 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:13.569 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:13.570 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:13.570 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:13.570 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:13.571 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:13.571 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:13.572 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:13.572 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:13.572 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:13.573 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:13.573 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:13.573 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:13.573 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:14.315 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:15.316 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:16.319 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:17.320 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:17.571 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:17.573 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:17.573 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:17.573 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:17.574 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:17.574 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:17.574 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:17.575 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:17.575 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:17.576 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:17.576 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:17.576 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:17.577 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:17.577 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:18.322 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:19.325 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:20.232 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:09:20.327 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:21.328 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:21.573 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:21.574 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:21.574 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:21.575 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:21.575 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:21.575 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:21.575 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:21.575 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:21.575 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:21.576 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:21.576 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:21.576 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:21.576 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:21.576 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:22.331 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:23.332 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:24.334 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:25.336 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:25.576 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:25.578 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:25.579 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:25.579 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:25.579 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:25.580 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:25.580 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:25.581 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:25.582 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:25.582 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:25.582 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:25.583 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:25.583 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:25.583 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:26.339 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:27.341 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:28.343 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:29.345 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:29.579 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:29.581 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:29.581 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:29.581 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:29.582 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:29.582 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:29.582 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:29.583 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:29.583 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:29.584 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:29.584 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:29.584 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:29.584 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:29.585 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:30.347 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:31.350 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:32.352 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:33.353 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:33.466 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:09:33.582 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:33.584 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:33.584 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:33.584 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:33.585 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:33.585 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:33.585 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:33.586 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:33.586 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:33.586 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:33.587 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:33.587 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:33.587 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:33.587 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:34.354 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:35.356 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:36.358 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:37.359 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:37.584 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:37.587 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:37.588 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:37.588 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:37.588 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:37.589 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:37.589 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:37.590 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:37.590 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:37.590 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:37.590 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:37.591 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:37.591 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:37.591 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:38.361 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:39.363 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:40.366 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:41.368 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:41.588 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:41.589 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:41.589 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:41.590 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:41.590 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:41.590 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:41.590 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:41.590 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:41.591 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:41.591 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:41.591 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:41.591 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:41.592 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:41.592 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:42.370 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:43.372 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:44.374 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:45.376 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:45.590 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:45.592 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:45.593 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:45.593 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:45.594 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:45.594 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:45.594 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:45.595 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:45.595 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:45.596 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:45.596 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:45.596 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:45.597 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:45.597 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:46.378 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:46.418 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:09:47.379 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:48.381 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:49.383 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:49.594 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:49.596 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:49.597 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:49.597 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:49.598 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:49.598 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:49.598 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:49.599 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:49.600 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:49.600 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:49.600 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:49.601 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:49.601 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:49.601 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:50.385 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:51.387 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:52.389 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:53.391 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:53.597 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:53.599 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:53.600 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:53.600 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:53.601 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:53.601 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:53.602 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:53.603 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:53.603 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:53.604 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:53.604 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:53.604 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:53.605 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:53.605 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:54.394 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:55.395 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:56.397 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:57.399 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:57.600 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:57.602 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:57.602 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:57.602 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:57.602 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:57.602 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:57.602 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:57.603 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:57.603 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:57.603 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:57.603 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:57.603 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:57.603 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:57.604 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:58.400 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:59.402 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:59.412 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:10:00.403 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:01.405 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:01.603 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:01.604 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:01.604 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:01.604 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:01.605 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:01.605 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:01.605 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:01.605 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:01.605 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:01.606 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:01.606 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:01.606 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:01.606 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:01.606 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:02.407 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:03.408 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:04.411 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:05.413 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:05.605 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:05.608 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:05.608 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:05.609 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:05.609 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:05.609 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:05.610 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:05.610 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:05.611 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:05.611 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:05.612 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:05.612 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:05.612 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:05.613 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:06.415 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:07.417 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:08.419 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:09.421 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:09.608 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:09.610 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:09.610 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:09.610 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:09.611 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:09.611 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:09.611 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:09.612 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:09.612 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:09.612 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:09.612 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:09.612 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:09.613 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:09.613 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:10.423 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:11.424 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:12.385 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:10:12.426 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:13.427 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:13.611 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:13.613 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:13.614 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:13.614 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:13.614 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:13.615 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:13.615 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:13.616 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:13.616 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:13.617 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:13.617 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:13.617 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:13.617 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:13.617 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:14.430 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:15.431 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:16.434 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:17.436 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:17.614 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:17.615 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:17.616 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:17.616 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:17.616 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:17.616 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:17.617 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:17.617 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:17.617 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:17.618 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:17.618 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:17.618 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:17.618 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:17.618 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:18.438 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:19.440 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:20.443 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:21.445 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:21.617 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:21.620 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:21.620 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:21.620 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:21.621 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:21.621 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:21.621 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:21.622 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:21.622 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:21.622 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:21.623 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:21.623 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:21.623 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:21.623 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:22.447 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:23.448 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:24.450 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:25.263 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:10:25.452 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:25.620 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:25.622 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:25.623 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:25.623 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:25.623 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:25.623 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:25.624 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:25.624 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:25.625 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:25.625 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:25.625 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:25.625 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:25.626 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:25.626 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:26.453 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:27.456 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:28.459 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:29.461 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:29.623 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:29.625 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:29.626 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:29.626 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:29.626 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:29.626 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:29.627 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:29.627 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:29.628 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:29.628 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:29.628 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:29.629 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:29.629 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:29.629 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:30.462 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:31.465 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:32.467 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:33.469 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:33.626 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:33.628 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:33.629 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:33.629 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:33.629 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:33.630 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:33.630 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:33.631 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:33.631 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:33.631 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:33.632 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:33.632 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:33.632 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:33.632 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:34.472 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:35.474 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:36.476 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:37.479 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:37.629 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:37.631 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:37.631 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:37.632 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:37.632 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:37.633 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:37.633 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:37.634 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:37.634 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:37.635 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:37.635 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:37.635 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:37.636 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:37.636 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:38.481 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:38.600 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:10:39.483 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:40.486 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:41.488 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:41.632 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:41.635 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:41.635 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:41.636 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:41.636 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:41.637 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:41.637 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:41.638 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:41.638 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:41.639 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:41.639 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:41.639 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:41.640 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:41.640 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:42.488 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:43.490 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:44.491 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:45.494 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:45.636 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:45.638 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:45.638 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:45.639 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:45.639 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:45.639 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:45.640 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:45.640 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:45.641 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:45.641 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:45.641 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:45.641 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:45.642 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:45.642 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:46.496 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:47.498 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:48.500 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:49.501 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:49.639 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:49.641 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:49.642 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:49.642 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:49.642 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:49.643 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:49.643 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:49.643 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:49.644 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:49.644 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:49.644 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:49.645 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:49.645 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:49.645 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:50.503 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:51.486 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:10:51.504 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:52.506 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:53.508 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:53.642 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:53.644 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:53.644 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:53.644 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:53.644 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:53.644 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:53.644 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:53.644 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:53.645 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:53.645 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:53.645 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:53.645 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:53.645 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:53.645 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:54.509 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:55.511 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:56.513 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:57.514 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:57.644 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:57.646 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:57.646 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:57.646 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:57.647 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:57.647 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:57.647 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:57.648 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:57.648 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:57.649 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:57.649 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:57.649 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:57.649 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:57.649 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:58.516 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:59.518 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:00.519 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:01.521 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:01.646 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:01.648 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:01.648 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:01.648 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:01.649 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:01.649 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:01.649 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:01.649 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:01.650 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:01.650 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:01.650 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:01.650 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:01.650 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:01.650 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:02.524 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:03.526 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:04.211 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:11:04.528 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:05.530 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:05.648 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:05.652 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:05.652 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:05.652 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:05.653 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:05.653 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:05.653 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:05.654 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:05.654 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:05.654 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:05.655 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:05.655 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:05.655 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:05.655 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:06.531 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:07.533 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:08.535 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:09.536 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:09.652 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:09.654 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:09.655 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:09.655 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:09.656 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:09.656 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:09.656 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:09.657 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:09.657 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:09.658 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:09.658 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:09.658 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:09.658 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:09.658 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:10.539 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:11.540 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:12.542 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:13.544 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:13.655 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:13.657 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:13.657 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:13.657 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:13.658 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:13.658 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:13.658 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:13.658 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:13.659 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:13.659 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:13.659 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:13.659 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:13.659 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:13.659 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:14.546 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:15.548 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:16.550 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:17.122 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:11:17.552 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:17.658 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:17.660 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:17.661 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:17.661 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:17.661 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:17.662 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:17.662 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:17.663 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:17.663 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:17.663 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:17.663 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:17.664 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:17.664 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:17.664 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:18.553 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:19.556 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:20.558 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:21.559 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:21.662 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:21.664 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:21.664 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:21.665 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:21.665 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:21.665 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:21.666 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:21.666 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:21.667 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:21.667 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:21.667 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:21.667 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:21.668 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:21.668 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:22.561 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:23.563 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:24.566 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:25.568 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:25.664 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:25.667 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:25.667 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:25.668 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:25.668 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:25.669 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:25.669 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:25.670 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:25.670 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:25.671 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:25.671 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:25.671 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:25.672 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:25.672 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:26.571 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:27.572 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:28.573 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:29.575 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:29.667 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:29.669 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:29.669 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:29.669 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:29.669 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:29.669 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:29.669 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:29.670 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:29.670 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:29.670 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:29.670 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:29.670 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:29.670 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:29.670 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:29.952 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:11:30.577 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:31.579 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:32.581 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:33.582 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:33.669 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:33.670 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:33.670 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:33.670 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:33.671 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:33.671 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:33.671 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:33.671 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:33.671 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:33.671 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:33.672 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:33.672 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:33.672 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:33.672 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:34.584 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:35.586 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:36.587 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:37.589 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:37.671 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:37.673 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:37.673 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:37.673 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:37.674 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:37.674 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:37.674 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:37.675 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:37.675 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:37.675 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:37.676 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:37.676 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:37.676 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:37.676 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:38.592 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:39.594 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:40.596 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:41.598 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:41.680 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:41.682 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:41.683 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:41.684 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:41.684 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:41.685 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:41.685 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:41.686 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:41.686 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:41.686 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:41.686 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:41.687 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:41.687 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:41.687 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:42.599 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:43.207 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:11:43.600 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:44.602 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:45.604 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:45.683 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:45.685 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:45.685 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:45.685 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:45.686 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:45.686 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:45.686 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:45.687 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:45.687 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:45.688 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:45.688 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:45.688 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:45.688 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:45.688 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:46.606 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:47.608 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:48.609 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:49.611 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:49.685 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:49.688 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:49.689 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:49.689 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:49.689 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:49.690 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:49.690 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:49.691 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:49.691 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:49.691 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:49.691 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:49.692 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:49.692 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:49.692 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:50.613 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:51.615 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:52.617 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:53.619 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:53.689 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:53.691 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:53.691 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:53.692 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:53.692 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:53.692 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:53.692 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:53.693 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:53.694 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:53.694 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:53.695 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:53.695 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:53.695 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:53.695 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:54.620 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:55.622 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:56.230 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:11:56.624 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:57.625 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:57.692 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:57.694 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:57.694 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:57.694 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:57.695 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:57.695 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:57.695 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:57.696 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:57.696 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:57.696 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:57.697 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:57.697 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:57.697 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:57.697 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:58.627 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:59.630 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:00.632 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:01.634 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:01.695 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:01.697 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:01.697 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:01.698 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:01.698 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:01.698 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:01.698 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:01.699 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:01.699 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:01.700 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:01.700 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:01.700 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:01.700 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:01.701 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:02.636 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:03.638 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:04.639 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:05.641 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:05.698 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:05.699 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:05.699 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:05.699 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:05.700 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:05.700 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:05.700 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:05.701 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:05.701 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:05.701 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:05.701 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:05.701 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:05.701 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:05.702 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:06.642 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:07.644 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:08.646 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:09.485 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:12:09.648 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:09.700 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:09.701 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:09.701 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:09.701 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:09.702 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:09.702 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:09.702 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:09.702 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:09.702 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:09.702 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:09.702 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:09.702 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:09.703 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:09.703 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:10.649 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:11.651 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:12.653 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:13.655 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:13.702 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:13.704 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:13.704 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:13.704 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:13.705 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:13.705 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:13.705 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:13.706 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:13.706 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:13.707 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:13.707 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:13.707 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:13.707 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:13.708 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:14.656 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:15.658 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:16.659 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:17.661 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:17.705 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:17.707 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:17.707 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:17.707 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:17.707 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:17.708 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:17.708 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:17.708 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:17.709 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:17.709 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:17.709 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:17.709 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:17.710 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:17.710 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:18.663 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:19.664 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:20.667 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:21.668 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:21.707 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:21.709 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:21.709 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:21.710 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:21.710 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:21.710 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:21.710 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:21.711 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:21.711 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:21.712 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:21.712 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:21.712 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:21.713 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:21.713 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:22.540 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:12:22.669 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:23.670 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:24.671 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:25.673 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:25.709 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:25.711 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:25.712 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:25.712 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:25.712 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:25.713 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:25.713 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:25.714 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:25.714 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:25.714 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:25.714 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:25.715 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:25.715 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:25.715 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:26.675 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:27.676 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:28.678 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:29.680 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:29.712 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:29.714 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:29.715 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:29.715 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:29.715 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:29.716 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:29.716 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:29.717 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:29.717 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:29.717 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:29.717 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:29.718 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:29.718 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:29.718 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:30.682 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:31.684 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:32.686 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:33.687 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:33.715 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:33.718 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:33.718 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:33.719 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:33.719 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:33.719 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:33.720 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:33.720 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:33.721 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:33.721 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:33.721 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:33.722 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:33.722 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:33.722 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:34.690 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:35.544 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:12:35.692 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:36.693 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:37.694 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:37.718 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:37.721 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:37.721 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:37.721 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:37.721 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:37.721 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:37.722 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:37.722 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:37.722 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:37.722 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:37.722 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:37.723 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:37.723 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:37.723 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:38.697 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:39.698 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:40.700 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:41.702 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:41.721 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:41.723 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:41.723 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:41.724 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:41.724 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:41.725 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:41.725 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:41.725 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:41.726 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:41.726 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:41.726 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:41.726 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:41.727 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:41.727 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:42.704 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:43.707 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:44.708 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:45.710 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:45.724 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:45.726 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:45.726 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:45.726 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:45.727 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:45.727 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:45.727 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:45.728 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:45.728 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:45.728 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:45.728 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:45.729 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:45.729 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:45.729 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:46.711 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:47.712 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:48.714 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:48.959 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:12:49.716 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:49.726 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:49.728 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:49.728 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:49.729 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:49.729 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:49.729 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:49.729 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:49.730 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:49.730 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:49.731 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:49.731 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:49.731 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:49.731 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:49.731 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:50.717 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:51.718 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:52.720 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:53.723 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:53.728 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:53.730 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:53.731 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:53.731 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:53.731 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:53.731 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:53.732 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:53.732 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:53.733 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:53.733 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:53.733 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:53.733 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:53.734 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:53.734 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:54.725 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:55.727 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:56.729 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:57.731 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:57.731 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:57.733 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:57.734 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:57.734 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:57.734 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:57.734 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:57.735 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:57.735 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:57.735 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:57.735 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:57.736 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:57.736 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:57.736 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:57.736 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:58.734 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:59.735 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:00.736 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:01.735 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:01.737 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:01.738 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:01.738 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:01.739 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:01.739 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:01.739 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:01.739 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:01.740 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:01.740 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:01.741 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:01.741 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:01.741 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:01.741 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:01.741 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:01.832 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:13:02.739 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:03.740 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:04.743 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:05.740 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:05.742 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:05.742 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:05.743 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:05.743 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:05.743 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:05.743 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:05.744 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:05.745 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:05.745 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:05.745 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:05.745 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:05.746 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:05.746 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:05.746 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:06.748 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:07.750 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:08.752 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:09.743 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:09.746 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:09.747 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:09.747 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:09.748 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:09.748 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:09.748 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:09.749 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:09.750 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:09.750 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:09.751 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:09.751 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:09.751 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:09.752 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:09.753 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:10.755 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:11.757 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:12.759 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:13.748 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:13.750 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:13.751 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:13.751 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:13.751 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:13.752 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:13.752 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:13.753 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:13.753 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:13.753 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:13.753 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:13.753 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:13.754 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:13.754 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:13.760 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:14.762 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:14.936 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:13:15.763 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:16.765 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:17.751 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:17.754 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:17.755 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:17.755 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:17.755 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:17.755 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:17.755 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:17.756 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:17.756 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:17.757 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:17.757 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:17.757 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:17.757 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:17.757 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:17.766 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:18.768 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:19.770 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:20.771 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:21.755 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:21.757 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:21.757 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:21.757 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:21.758 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:21.758 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:21.758 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:21.759 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:21.759 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:21.759 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:21.759 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:21.759 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:21.760 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:21.760 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:21.772 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:22.774 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:23.776 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:24.778 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:25.758 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:25.760 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:25.760 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:25.760 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:25.761 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:25.761 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:25.761 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:25.762 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:25.762 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:25.763 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:25.763 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:25.763 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:25.763 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:25.764 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:25.779 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:26.780 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:27.781 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:27.917 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:13:28.783 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:29.762 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:29.764 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:29.764 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:29.764 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:29.765 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:29.765 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:29.765 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:29.766 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:29.766 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:29.766 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:29.767 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:29.767 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:29.767 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:29.767 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:29.785 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:30.787 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:31.789 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:32.791 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:33.765 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:33.767 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:33.767 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:33.768 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:33.768 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:33.768 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:33.769 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:33.769 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:33.770 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:33.770 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:33.770 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:33.770 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:33.771 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:33.771 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:33.794 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:34.796 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:35.798 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:36.800 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:37.768 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:37.769 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:37.770 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:37.770 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:37.770 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:37.770 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:37.770 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:37.771 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:37.771 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:37.771 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:37.772 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:37.772 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:37.772 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:37.772 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:37.802 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:38.804 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:39.806 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:40.808 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:41.051 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:13:41.771 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:41.773 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:41.774 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:41.774 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:41.774 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:41.774 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:41.775 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:41.775 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:41.776 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:41.776 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:41.777 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:41.777 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:41.777 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:41.777 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:41.809 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:42.810 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:43.812 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:44.814 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:45.774 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:45.776 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:45.777 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:45.777 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:45.777 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:45.778 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:45.778 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:45.778 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:45.779 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:45.779 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:45.779 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:45.779 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:45.780 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:45.780 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:45.816 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:46.818 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:47.819 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:48.821 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:49.778 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:49.780 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:49.781 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:49.781 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:49.782 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:49.784 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:49.784 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:49.785 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:49.785 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:49.785 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:49.786 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:49.786 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:49.786 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:49.786 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:49.822 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:50.824 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:51.825 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:52.826 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:53.781 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:53.784 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:53.784 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:53.784 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:53.785 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:53.785 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:53.786 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:53.786 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:53.787 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:53.787 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:53.787 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:53.787 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:53.788 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:53.788 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:53.827 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:54.383 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:13:54.829 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:55.831 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:56.833 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:57.785 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:57.786 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:57.786 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:57.787 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:57.787 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:57.787 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:57.787 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:57.788 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:57.788 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:57.788 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:57.788 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:57.789 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:57.789 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:57.789 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:57.834 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:58.836 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:59.838 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:00.840 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:01.788 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:01.790 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:01.790 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:01.791 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:01.791 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:01.791 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:01.791 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:01.792 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:01.792 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:01.792 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:01.792 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:01.792 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:01.793 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:01.793 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:01.841 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:02.843 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:03.845 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:04.847 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:05.791 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:05.793 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:05.793 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:05.793 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:05.793 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:05.793 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:05.793 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:05.793 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:05.793 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:05.794 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:05.794 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:05.794 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:05.794 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:05.794 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:05.848 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:06.850 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:07.214 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:14:07.852 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:08.854 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:09.794 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:09.796 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:09.796 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:09.797 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:09.797 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:09.797 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:09.797 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:09.798 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:09.799 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:09.799 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:09.799 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:09.799 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:09.800 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:09.800 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:09.855 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:10.857 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:11.858 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:12.859 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:13.797 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:13.799 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:13.800 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:13.800 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:13.800 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:13.801 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:13.801 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:13.802 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:13.802 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:13.802 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:13.803 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:13.803 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:13.803 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:13.803 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:13.861 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:14.863 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:15.864 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:16.866 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:17.800 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:17.803 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:17.803 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:17.803 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:17.804 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:17.804 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:17.804 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:17.805 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:17.805 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:17.806 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:17.806 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:17.806 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:17.807 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:17.807 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:17.867 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:18.870 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:19.871 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:20.319 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:14:20.873 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:21.804 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:21.806 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:21.807 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:21.807 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:21.808 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:21.808 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:21.808 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:21.809 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:21.809 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:21.810 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:21.810 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:21.810 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:21.810 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:21.810 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:21.874 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:22.875 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:23.876 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:24.878 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:25.808 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:25.810 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:25.811 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:25.811 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:25.811 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:25.812 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:25.812 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:25.813 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:25.813 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:25.813 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:25.814 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:25.814 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:25.814 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:25.814 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:25.880 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:26.882 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:27.883 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:28.885 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:29.812 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:29.814 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:29.814 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:29.814 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:29.815 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:29.815 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:29.815 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:29.816 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:29.816 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:29.817 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:29.817 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:29.817 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:29.817 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:29.817 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:29.887 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:30.889 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:31.891 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:32.892 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:33.358 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:14:33.815 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:33.816 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:33.817 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:33.817 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:33.817 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:33.817 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:33.817 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:33.818 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:33.818 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:33.818 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:33.818 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:33.819 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:33.819 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:33.819 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:33.893 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:34.896 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:35.898 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:36.900 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:37.818 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:37.820 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:37.821 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:37.821 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:37.821 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:37.821 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:37.822 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:37.822 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:37.822 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:37.823 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:37.823 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:37.823 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:37.823 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:37.824 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:37.901 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:38.903 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:39.905 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:40.907 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:41.822 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:41.823 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:41.823 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:41.824 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:41.824 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:41.824 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:41.824 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:41.825 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:41.825 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:41.825 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:41.825 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:41.825 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:41.825 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:41.825 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:41.909 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:42.911 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:43.912 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:44.915 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:45.824 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:45.827 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:45.827 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:45.828 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:45.828 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:45.828 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:45.828 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:45.829 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:45.829 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:45.830 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:45.830 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:45.830 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:45.830 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:45.831 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:45.916 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:46.212 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:14:46.919 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:47.921 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:48.923 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:49.828 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:49.830 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:49.830 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:49.831 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:49.831 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:49.831 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:49.831 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:49.832 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:49.833 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:49.833 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:49.833 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:49.833 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:49.833 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:49.834 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:49.924 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:50.926 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:51.928 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:52.929 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:53.832 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:53.834 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:53.834 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:53.834 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:53.834 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:53.834 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:53.834 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:53.835 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:53.835 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:53.835 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:53.835 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:53.835 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:53.836 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:53.836 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:53.931 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:54.933 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:55.935 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:56.936 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:57.835 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:57.836 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:57.837 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:57.837 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:57.837 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:57.837 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:57.837 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:57.838 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:57.838 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:57.838 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:57.838 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:57.838 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:57.838 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:57.838 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:57.938 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:58.940 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:59.827 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:14:59.942 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:00.943 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:01.839 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:01.842 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:01.842 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:01.842 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:01.843 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:01.843 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:01.843 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:01.843 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:01.844 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:01.844 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:01.844 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:01.844 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:01.844 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:01.844 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:01.945 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:02.947 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:03.949 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:04.950 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:05.843 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:05.845 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:05.845 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:05.846 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:05.846 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:05.846 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:05.846 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:05.847 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:05.847 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:05.848 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:05.848 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:05.849 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:05.849 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:05.849 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:05.952 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:06.954 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:07.956 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:08.958 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:09.846 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:09.848 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:09.849 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:09.849 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:09.849 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:09.850 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:09.850 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:09.851 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:09.851 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:09.851 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:09.852 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:09.852 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:09.852 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:09.852 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:09.959 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:10.961 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:11.963 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:12.939 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:15:12.964 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:13.850 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:13.851 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:13.851 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:13.852 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:13.852 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:13.852 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:13.852 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:13.853 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:13.853 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:13.853 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:13.854 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:13.854 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:13.854 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:13.854 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:13.966 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:14.967 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:15.969 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:16.971 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:17.852 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:17.853 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:17.853 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:17.854 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:17.854 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:17.854 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:17.854 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:17.854 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:17.854 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:17.855 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:17.855 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:17.855 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:17.855 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:17.855 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:17.972 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:18.975 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:19.977 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:20.978 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:21.855 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:21.856 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:21.856 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:21.856 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:21.856 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:21.856 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:21.857 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:21.857 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:21.857 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:21.857 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:21.857 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:21.857 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:21.857 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:21.858 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:21.980 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:22.982 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:23.984 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:24.985 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:25.822 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:15:25.857 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:25.859 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:25.860 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:25.860 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:25.860 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:25.861 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:25.861 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:25.862 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:25.862 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:25.862 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:25.862 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:25.862 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:25.863 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:25.863 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:25.986 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:26.988 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:27.989 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:28.991 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:29.860 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:29.863 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:29.864 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:29.864 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:29.864 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:29.865 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:29.865 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:29.865 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:29.866 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:29.866 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:29.866 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:29.866 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:29.867 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:29.867 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:29.993 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:30.995 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:31.997 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:32.999 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:33.864 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:33.866 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:33.867 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:33.867 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:33.867 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:33.867 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:33.867 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:33.868 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:33.868 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:33.869 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:33.869 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:33.869 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:33.870 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:33.870 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:34.000 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:35.002 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:36.004 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:37.005 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:37.867 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:37.869 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:37.870 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:37.870 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:37.870 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:37.870 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:37.870 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:37.871 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:37.871 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:37.872 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:37.872 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:37.872 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:37.872 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:37.872 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:38.007 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:38.735 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:15:39.009 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:40.011 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:41.014 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:41.870 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:41.872 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:41.873 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:41.873 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:41.873 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:41.873 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:41.874 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:41.874 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:41.875 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:41.875 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:41.875 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:41.876 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:41.876 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:41.876 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:42.015 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:43.016 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:44.018 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:45.020 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:45.873 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:45.876 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:45.876 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:45.877 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:45.877 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:45.877 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:45.877 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:45.878 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:45.878 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:45.879 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:45.879 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:45.879 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:45.880 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:45.880 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:46.024 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:47.027 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:48.028 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:49.030 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:49.877 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:49.879 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:49.880 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:49.880 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:49.881 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:49.881 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:49.882 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:49.883 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:49.883 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:49.884 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:49.884 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:49.885 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:49.885 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:49.885 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:50.031 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:51.034 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:51.790 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:15:52.035 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:53.048 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:53.880 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:53.882 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:53.883 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:53.883 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:53.883 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:53.883 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:53.884 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:53.884 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:53.885 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:53.885 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:53.885 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:53.886 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:53.886 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:53.886 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:54.049 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:55.051 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:56.053 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:57.055 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:57.883 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:57.884 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:57.885 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:57.885 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:57.885 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:57.885 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:57.885 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:57.886 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:57.886 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:57.886 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:57.886 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:57.886 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:57.886 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:57.886 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:58.057 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:59.059 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:00.060 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:01.061 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:01.886 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:01.888 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:01.889 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:01.889 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:01.889 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:01.890 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:01.890 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:01.890 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:01.891 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:01.891 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:01.891 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:01.891 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:01.892 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:01.892 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:02.063 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:03.064 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:04.066 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:05.068 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:05.219 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:16:05.890 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:05.892 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:05.892 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:05.893 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:05.893 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:05.893 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:05.893 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:05.894 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:05.894 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:05.895 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:05.895 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:05.895 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:05.895 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:05.895 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:06.070 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:07.072 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:08.073 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:09.075 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:09.893 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:09.896 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:09.896 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:09.897 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:09.897 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:09.898 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:09.898 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:09.899 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:09.899 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:09.900 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:09.900 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:09.901 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:09.901 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:09.901 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:10.077 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:11.078 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:12.079 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:13.080 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:13.897 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:13.900 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:13.900 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:13.900 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:13.901 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:13.901 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:13.901 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:13.902 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:13.902 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:13.903 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:13.903 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:13.903 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:13.903 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:13.903 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:14.082 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:15.084 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:16.085 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:17.086 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:17.902 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:17.903 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:17.904 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:17.904 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:17.904 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:17.905 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:17.905 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:17.905 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:17.906 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:17.906 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:17.906 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:17.906 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:17.906 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:17.906 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:18.088 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:18.093 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:16:19.090 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:20.092 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:21.093 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:21.904 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:21.906 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:21.907 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:21.907 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:21.907 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:21.908 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:21.908 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:21.909 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:21.909 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:21.910 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:21.910 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:21.910 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:21.910 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:21.911 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:22.094 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:23.096 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:24.098 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:25.100 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:25.907 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:25.910 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:25.910 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:25.910 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:25.911 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:25.911 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:25.911 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:25.912 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:25.912 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:25.913 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:25.913 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:25.913 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:25.914 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:25.914 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:26.101 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:27.103 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:28.104 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:29.106 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:29.911 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:29.912 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:29.912 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:29.912 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:29.913 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:29.913 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:29.913 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:29.913 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:29.913 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:29.913 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:29.913 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:29.913 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:29.913 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:29.914 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:30.108 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:31.045 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:16:31.110 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:32.112 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:33.114 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:33.913 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:33.915 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:33.916 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:33.916 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:33.916 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:33.917 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:33.917 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:33.918 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:33.918 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:33.918 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:33.918 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:33.919 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:33.919 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:33.919 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:34.116 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:35.118 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:36.119 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:37.120 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:37.917 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:37.919 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:37.920 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:37.920 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:37.920 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:37.921 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:37.921 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:37.922 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:37.922 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:37.922 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:37.922 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:37.923 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:37.923 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:37.923 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:38.122 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:39.124 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:40.125 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:41.127 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:41.920 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:41.922 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:41.923 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:41.923 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:41.923 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:41.924 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:41.924 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:41.925 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:41.925 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:41.925 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:41.926 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:41.926 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:41.926 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:41.926 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:42.128 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:43.129 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:44.069 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:16:44.131 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:45.133 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:45.924 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:45.926 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:45.926 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:45.926 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:45.927 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:45.927 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:45.927 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:45.928 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:45.928 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:45.929 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:45.929 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:45.929 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:45.929 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:45.929 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:46.135 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:47.138 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:48.140 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:49.141 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:49.927 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:49.929 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:49.929 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:49.929 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:49.929 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:49.930 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:49.930 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:49.930 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:49.931 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:49.931 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:49.931 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:49.931 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:49.931 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:49.931 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:50.143 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:51.145 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:52.147 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:53.148 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:53.929 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:53.931 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:53.931 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:53.932 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:53.932 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:53.933 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:53.933 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:53.934 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:53.934 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:53.934 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:53.934 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:53.935 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:53.935 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:53.935 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:54.150 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:55.152 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:56.154 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:57.058 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:16:57.155 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:57.931 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:57.934 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:57.935 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:57.935 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:57.935 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:57.936 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:57.936 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:57.937 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:57.937 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:57.937 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:57.938 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:57.938 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:57.938 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:57.938 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:58.157 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:59.159 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:00.161 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:01.163 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:01.936 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:01.937 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:01.937 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:01.937 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:01.937 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:01.937 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:01.937 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:01.938 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:01.938 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:01.938 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:01.938 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:01.938 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:01.938 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:01.938 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:02.165 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:03.167 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:04.169 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:05.171 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:05.938 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:05.940 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:05.941 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:05.941 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:05.941 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:05.942 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:05.942 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:05.943 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:05.943 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:05.944 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:05.944 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:05.944 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:05.945 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:05.945 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:06.173 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:07.175 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:08.177 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:09.178 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:09.941 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:09.942 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:09.942 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:09.942 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:09.943 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:09.943 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:09.943 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:09.943 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:09.943 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:09.944 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:09.944 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:09.944 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:09.944 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:09.944 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:10.180 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:10.500 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:17:11.181 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:12.183 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:13.184 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:13.489 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:17:13.943 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:13.946 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:13.946 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:13.946 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:13.947 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:13.947 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:13.947 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:13.948 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:13.948 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:13.949 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:13.949 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:13.949 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:13.950 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:13.950 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:14.186 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:15.188 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:16.190 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:16.413 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:17:17.192 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:17.947 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:17.949 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:17.950 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:17.950 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:17.950 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:17.951 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:17.951 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:17.952 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:17.952 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:17.952 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:17.953 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:17.953 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:17.953 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:17.953 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:18.193 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:19.195 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:20.196 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:21.198 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:21.950 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:21.952 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:21.952 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:21.952 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:21.953 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:21.953 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:21.953 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:21.954 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:21.954 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:21.954 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:21.955 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:21.955 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:21.955 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:21.955 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:22.200 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:23.201 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:24.203 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:25.206 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:25.953 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:25.954 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:25.955 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:25.955 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:25.955 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:25.955 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:25.955 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:25.956 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:25.956 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:25.956 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:25.956 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:25.956 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:25.956 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:25.957 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:26.225 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:27.226 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:28.228 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:29.230 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:29.956 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:29.958 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:29.958 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:29.958 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:29.959 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:29.959 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:29.959 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:29.960 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:29.960 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:29.961 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:29.961 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:29.961 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:29.961 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:29.961 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:30.232 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:31.235 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:32.237 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:33.239 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:33.958 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:33.960 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:33.960 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:33.960 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:33.960 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:33.960 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:33.960 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:33.961 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:33.961 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:33.961 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:33.961 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:33.961 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:33.961 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:33.961 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:34.240 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:35.241 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:36.242 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:37.245 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:37.961 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:37.962 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:37.962 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:37.962 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:37.963 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:37.963 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:37.963 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:37.963 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:37.963 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:37.963 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:37.963 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:37.964 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:37.964 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:37.964 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:38.247 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:39.248 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:40.249 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:41.251 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:41.964 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:41.966 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:41.966 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:41.966 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:41.967 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:41.967 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:41.967 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:41.968 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:41.968 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:41.968 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:41.968 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:41.969 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:41.969 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:41.969 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:42.253 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:43.254 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:44.256 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:45.258 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:45.967 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:45.970 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:45.971 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:45.971 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:45.971 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:45.971 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:45.972 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:45.972 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:45.972 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:45.973 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:45.973 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:45.973 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:45.973 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:45.973 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:46.260 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:47.261 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:47.572 7f8fa80c5c20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:17:48.263 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:49.265 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:49.972 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:49.974 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:49.974 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:49.974 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:49.975 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:49.975 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:49.975 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:49.976 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:49.976 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:49.976 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:49.977 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:49.977 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:49.977 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:49.977 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:50.266 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:51.267 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:52.269 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:53.272 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:53.974 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:53.977 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:53.977 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:53.977 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:53.978 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:53.978 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:53.978 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:53.979 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:53.979 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:53.979 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:53.980 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:53.980 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:53.980 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:53.980 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:54.274 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:55.276 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:56.279 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:57.280 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:57.977 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:57.979 7f8fa80c5c20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:57.979 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:57.979 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:57.980 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:57.980 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:57.980 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:57.981 7f8fa80c5c20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:57.981 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:57.981 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.56:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.56:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.56:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.56:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:57.981 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:57.981 7f8fa80c5c20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:57.982 7f8fa80c5c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 109 http_port = 8096 start_time = 1756691837027098 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:57.982 7f8fa80c5c20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:58.283 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:59.290 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:00.291 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:01.292 7f8fa95aa540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process.