09/01/2025 01:56:35.325 7f6076b279c0 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/01/2025 01:56:35.325 7f6076b279c0 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/01/2025 01:56:35.325 7f6076b279c0 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/01/2025 01:56:35.325 7f6071988040 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 09/01/2025 01:56:35.325 7f6071988040 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:56:35.326 7f6071988040 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/01/2025 01:56:35.326 7f6071988040 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/01/2025 01:56:35.326 7f6071988040 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:35.326 7f6071988040 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/01/2025 01:56:35.326 7f6071988040 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:35.326 7f6071988040 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:35.326 7f6071988040 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 start_time = 1756691795326032 collector_name = collector_ip = collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/01/2025 01:56:35.326 7f6071988040 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:35.350 7f6076b279c0 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.249:5673//', 'pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.131:5673//'] 09/01/2025 01:56:35.357 7f6076b279c0 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 09/01/2025 01:56:35.357 7f60719889a0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:35.358 7f60719889a0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 09/01/2025 01:56:35.358 7f60719889a0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:35.358 7f60719889a0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:35.358 7f60719889a0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 09/01/2025 01:56:35.358 7f60719889a0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:35.387 7f6076b279c0 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/01/2025 01:56:35.388 7f6073fde4a0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:35.388 7f6073fde4a0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/01/2025 01:56:35.388 7f6073fde4a0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:35.408 7f6076b279c0 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 09/01/2025 01:56:35.408 7f60717240e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:35.408 7f60717240e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 09/01/2025 01:56:35.409 7f60717240e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:35.411 7f6076b279c0 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/01/2025 01:56:35.412 7f60717240e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:35.412 7f60717240e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/01/2025 01:56:35.412 7f60717240e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:35.431 7f6076b279c0 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 09/01/2025 01:56:35.431 7f6076b279c0 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 09/01/2025 01:56:35.432 7f6076b279c0 [contrail-device-manager] [WARNING]: initializing cassandra cluster 09/01/2025 01:56:35.432 7f6076b279c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/01/2025 01:56:35.432 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 09/01/2025 01:56:35.432 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 09/01/2025 01:56:35.433 7f6071724680 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 09/01/2025 01:56:35.433 7f6071724680 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 09/01/2025 01:56:35.433 7f6071724900 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:35.434 7f6071724900 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:56:35.434 7f6071724900 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:35.435 7f6076b279c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/01/2025 01:56:35.437 7f6071724540 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 09/01/2025 01:56:35.437 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 09/01/2025 01:56:35.439 7f6071724680 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 09/01/2025 01:56:35.499 7f6076b279c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/01/2025 01:56:35.500 7f6071728cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:35.500 7f6071728cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/01/2025 01:56:35.500 7f6071728cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:35.507 7f6076b279c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 09/01/2025 01:56:35.507 7f60717289a0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:35.507 7f60717289a0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:56:35.507 7f60717289a0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:35.518 7f6076b279c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 09/01/2025 01:56:35.540 7f6076b279c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 09/01/2025 01:56:36.485 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:56:37.471 7f6076b279c0 [contrail-device-manager] [WARNING]: table 'obj_uuid_table', created 09/01/2025 01:56:37.475 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:56:38.608 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:56:39.326 7f6071afa360 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:39.327 7f6071afa360 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:39.327 7f6071afa360 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:39.327 7f6071afa360 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:39.327 7f6071afa360 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:56:39.327 7f6071afa360 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:39.327 7f6071afa360 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:39.327 7f6071afa360 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:39.327 7f6071afa360 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:39.328 7f6071afa360 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:56:39.328 7f6071afa360 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:39.328 7f6071afa360 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:39.328 7f6071afa360 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:56:39.328 7f6071afa360 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:39.522 7f6076b279c0 [contrail-device-manager] [WARNING]: table 'obj_uuid_table' fixed with properties {'gc_grace_seconds': 864000} 09/01/2025 01:56:39.526 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:56:41.594 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:56:43.268 7f6076b279c0 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table', created 09/01/2025 01:56:43.272 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:56:43.327 7f6072445cc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:43.335 7f6072445cc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:43.336 7f6072445cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:43.336 7f6072445cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:43.336 7f6072445cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:56:43.336 7f6072445cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:43.336 7f6072445cc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:43.336 7f6072445cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:43.336 7f6072445cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:43.337 7f6072445cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:56:43.337 7f6072445cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:43.337 7f6072445cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:43.337 7f6072445cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:56:43.337 7f6072445cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:44.696 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:56:45.614 7f6076b279c0 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table' fixed with properties {'gc_grace_seconds': 864000} 09/01/2025 01:56:45.618 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:56:47.336 7f6071afa360 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:47.338 7f6071afa360 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:47.338 7f6071afa360 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:47.338 7f6071afa360 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:47.338 7f6071afa360 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:56:47.338 7f6071afa360 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:47.338 7f6071afa360 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:47.339 7f6071afa360 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:47.339 7f6071afa360 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:47.339 7f6071afa360 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:56:47.339 7f6071afa360 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:47.339 7f6071afa360 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:47.339 7f6071afa360 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:56:47.339 7f6071afa360 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:47.559 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:56:47.572 7f6076b279c0 [contrail-device-manager] [WARNING]: table 'obj_shared_table', created 09/01/2025 01:56:47.577 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:56:49.459 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:56:50.581 7f6076b279c0 [contrail-device-manager] [WARNING]: table 'obj_shared_table' fixed with properties {'gc_grace_seconds': 864000} 09/01/2025 01:56:50.591 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:56:50.592 7f60724457c0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:50.592 7f60724457c0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:50.592 7f60724457c0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:50.655 7f6071988220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:50.656 7f6071988220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Zookeeper:Zookeeper[] connection down >>, ] >> 09/01/2025 01:56:50.656 7f6071988220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:50.669 7f6071988220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:50.669 7f6071988220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:50.669 7f6071988220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:50.669 7f6076b279c0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 09/01/2025 01:56:50.669 7f6076b279c0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 09/01/2025 01:56:50.670 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 09/01/2025 01:56:50.670 7f6076b279c0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3995-3. 09/01/2025 01:56:50.676 7f6071724540 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 09/01/2025 01:56:50.765 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3995-3.'] 09/01/2025 01:56:50.784 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 09/01/2025 01:56:50.795 7f6071724540 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 09/01/2025 01:56:50.820 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3995-3.'] 09/01/2025 01:56:51.182 7f6076b279c0 [contrail-device-manager] [DEBUG]: Fetching all containers 09/01/2025 01:56:51.217 7f6076b279c0 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 09/01/2025 01:56:51.338 7f6071a16f40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:51.339 7f6071a16f40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:51.339 7f6071a16f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:51.339 7f6071a16f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:51.340 7f6071a16f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:51.340 7f6071a16f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:51.340 7f6071a16f40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:51.340 7f6071a16f40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:51.340 7f6071a16f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:51.340 7f6071a16f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:51.340 7f6071a16f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:51.340 7f6071a16f40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:51.340 7f6071a16f40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:56:51.340 7f6071a16f40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:51.415 7f6076b279c0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3995-3..config_queue 09/01/2025 01:56:51.416 7f6076b279c0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3995-3..tftp_queue 09/01/2025 01:56:51.823 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:51.835 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 09/01/2025 01:56:51.839 7f6071724540 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 09/01/2025 01:56:51.913 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3995-3.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3995-3..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3995-3..tftp_queue'] 09/01/2025 01:56:51.917 7f6076b279c0 [contrail-device-manager] [WARNING]: Process 132 prepared to run in Full mode... 09/01/2025 01:56:51.917 7f6076b279c0 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 09/01/2025 01:56:51.917 7f6076b279c0 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 09/01/2025 01:56:51.923 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/local/lib/python3.9/site-packages/cfgm_common/uve/config_updater] to home page 09/01/2025 01:56:51.923 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 09/01/2025 01:56:51.925 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/local/lib/python3.9/site-packages/cfgm_common/uve/greenlets] to home page 09/01/2025 01:56:51.925 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 09/01/2025 01:56:51.926 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/local/lib/python3.9/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 09/01/2025 01:56:51.927 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 09/01/2025 01:56:51.934 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/local/lib/python3.9/site-packages/device_manager/sandesh/dm_introspect] to home page 09/01/2025 01:56:51.934 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 09/01/2025 01:56:51.934 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 09/01/2025 01:56:51.934 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 09/01/2025 01:56:51.935 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 09/01/2025 01:56:51.935 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 09/01/2025 01:56:51.935 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 09/01/2025 01:56:51.935 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 09/01/2025 01:56:51.935 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 09/01/2025 01:56:51.935 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 09/01/2025 01:56:51.935 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 09/01/2025 01:56:51.935 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 09/01/2025 01:56:51.935 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 09/01/2025 01:56:51.938 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/local/lib/python3.9/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 09/01/2025 01:56:51.938 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 09/01/2025 01:56:51.938 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 09/01/2025 01:56:51.938 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 09/01/2025 01:56:51.939 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/local/lib/python3.9/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 09/01/2025 01:56:51.939 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 09/01/2025 01:56:51.940 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 09/01/2025 01:56:51.940 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/local/lib/python3.9/site-packages/pysandesh/gen_py/sandesh_trace] to home page 09/01/2025 01:56:51.941 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 09/01/2025 01:56:51.941 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 09/01/2025 01:56:51.941 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 09/01/2025 01:56:51.941 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 09/01/2025 01:56:51.941 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 09/01/2025 01:56:51.943 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/local/lib/python3.9/site-packages/pysandesh/gen_py/sandesh_uve] to home page 09/01/2025 01:56:51.943 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 09/01/2025 01:56:51.943 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 09/01/2025 01:56:51.943 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 09/01/2025 01:56:51.943 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 09/01/2025 01:56:51.944 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 09/01/2025 01:56:51.944 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 09/01/2025 01:56:51.944 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 09/01/2025 01:56:51.944 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 09/01/2025 01:56:51.944 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 09/01/2025 01:56:51.945 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 09/01/2025 01:56:51.945 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 09/01/2025 01:56:51.945 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 09/01/2025 01:56:51.945 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 09/01/2025 01:56:51.945 7f6076b279c0 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 09/01/2025 01:56:51.948 7f6076b279c0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 09/01/2025 01:56:52.257 7f60718c8860 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 09/01/2025 01:56:52.257 7f60718c8860 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 09/01/2025 01:56:52.257 7f60705829a0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:52.258 7f60705829a0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 09/01/2025 01:56:52.258 7f60705829a0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:52.263 7f60705829a0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:52.263 7f60705829a0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 09/01/2025 01:56:52.263 7f60705829a0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:52.916 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:52.928 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 09/01/2025 01:56:52.934 7f6071724540 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 09/01/2025 01:56:52.980 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3995-3.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3995-3..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3995-3..tftp_queue', 'device_manager_ztp.ztp_queue'] 09/01/2025 01:56:53.983 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:54.985 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:55.340 7f6070582cc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:55.341 7f6070582cc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:55.341 7f6070582cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:55.341 7f6070582cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:55.342 7f6070582cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 09/01/2025 01:56:55.342 7f6070582cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:55.342 7f6070582cc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:55.342 7f6070582cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:55.342 7f6070582cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:55.342 7f6070582cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 09/01/2025 01:56:55.342 7f6070582cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:55.342 7f6070582cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:55.343 7f6070582cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:56:55.343 7f6070582cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:55.985 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:56.987 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:57.988 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:58.289 7f6076b279c0 [contrail-device-manager] [ERROR]: predef payloads file does not exist: /opt/contrail/fabric_ansible_playbooks/conf/predef_payloads.json 09/01/2025 01:56:58.289 7f6076b279c0 [contrail-device-manager] [ERROR]: Unable to load init data 09/01/2025 01:56:58.289 7f6076b279c0 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 09/01/2025 01:56:58.289 7f6076b279c0 [contrail-device-manager] [WARNING]: initializing cassandra cluster 09/01/2025 01:56:58.289 7f6076b279c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/01/2025 01:56:58.289 7f6070582cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:58.290 7f6070582cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:58.290 7f6070582cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:58.290 7f6070582cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:58.290 7f6070582cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:56:58.290 7f6070582cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:58.291 7f6076b279c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/01/2025 01:56:58.325 7f6076b279c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/01/2025 01:56:58.326 7f60701eec20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:58.326 7f60701eec20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Initializing description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/01/2025 01:56:58.326 7f60701eec20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:58.332 7f6076b279c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 09/01/2025 01:56:58.333 7f60701eec20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:58.333 7f60701eec20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:56:58.333 7f60701eec20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:58.491 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:56:58.990 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:59.342 7f6070130680 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:59.343 7f6070130680 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:59.343 7f6070130680 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:59.343 7f6070130680 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:59.343 7f6070130680 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:56:59.343 7f6070130680 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:59.343 7f6070130680 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:59.344 7f6070130680 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:59.344 7f6070130680 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:59.344 7f6070130680 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:56:59.344 7f6070130680 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:59.344 7f6070130680 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:59.344 7f6070130680 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:56:59.344 7f6070130680 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:59.568 7f6076b279c0 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace', created 09/01/2025 01:56:59.574 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:56:59.991 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:00.992 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:01.584 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:57:01.994 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:02.679 7f6076b279c0 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None} 09/01/2025 01:57:02.690 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:57:02.690 7f6076b279c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 09/01/2025 01:57:02.782 7f6076b279c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 09/01/2025 01:57:02.996 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:03.343 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:03.345 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:03.345 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:03.345 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:03.345 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:57:03.345 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:03.345 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:03.346 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:03.346 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:03.346 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:57:03.346 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:03.346 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:03.346 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:03.346 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:03.997 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:04.382 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:57:05.003 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:05.570 7f6076b279c0 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table', created 09/01/2025 01:57:05.574 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:57:06.008 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:06.486 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:57:07.009 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:07.346 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:07.348 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:07.348 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:07.348 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:07.349 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:57:07.349 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:07.349 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:07.349 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:07.349 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:07.349 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:57:07.349 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:07.349 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:07.349 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:07.350 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:07.599 7f6076b279c0 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table' fixed with properties {'gc_grace_seconds': 864000} 09/01/2025 01:57:07.604 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:57:08.010 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:08.600 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:57:09.011 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:09.552 7f6076b279c0 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table', created 09/01/2025 01:57:09.558 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:57:10.012 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:10.677 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:57:11.013 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:11.349 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:11.350 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:11.350 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:11.350 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:11.350 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:57:11.351 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:11.351 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:11.351 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:11.351 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:11.351 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:57:11.351 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:11.351 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:11.352 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:11.352 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:11.589 7f6076b279c0 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table' fixed with properties {'gc_grace_seconds': 864000} 09/01/2025 01:57:11.594 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:57:12.014 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:12.540 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:57:13.014 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:13.679 7f6076b279c0 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table', created 09/01/2025 01:57:13.685 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:57:14.015 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:14.611 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:57:15.017 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:15.350 7f6071728040 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:15.351 7f6071728040 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:15.352 7f6071728040 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:15.352 7f6071728040 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:15.352 7f6071728040 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:57:15.352 7f6071728040 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:15.352 7f6071728040 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:15.352 7f6071728040 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:15.352 7f6071728040 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:15.352 7f6071728040 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:57:15.353 7f6071728040 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:15.353 7f6071728040 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:15.353 7f6071728040 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:15.353 7f6071728040 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:15.492 7f6076b279c0 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table' fixed with properties {'gc_grace_seconds': 864000} 09/01/2025 01:57:15.498 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:57:16.018 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:17.019 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:17.305 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:57:18.020 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:18.482 7f6076b279c0 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table', created 09/01/2025 01:57:18.486 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:57:19.022 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:19.352 7f6071728180 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:19.354 7f6071728180 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:19.354 7f6071728180 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:19.354 7f6071728180 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:19.354 7f6071728180 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:57:19.354 7f6071728180 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:19.354 7f6071728180 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:19.354 7f6071728180 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:19.354 7f6071728180 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:19.355 7f6071728180 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:57:19.355 7f6071728180 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:19.355 7f6071728180 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:19.355 7f6071728180 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:19.355 7f6071728180 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:19.700 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:57:20.023 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:21.024 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:21.599 7f6076b279c0 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table' fixed with properties {'gc_grace_seconds': 864000} 09/01/2025 01:57:21.604 7f6076b279c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/01/2025 01:57:21.605 7f6071728040 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:21.605 7f6071728040 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:21.605 7f6071728040 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:21.643 7f6076b279c0 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.249:5673//', 'pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.131:5673//'] 09/01/2025 01:57:21.643 7f6076b279c0 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 09/01/2025 01:57:21.644 7f607060b7c0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:21.644 7f607060b7c0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Initializing description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 09/01/2025 01:57:21.644 7f607060b7c0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:21.644 7f607060b7c0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:21.644 7f607060b7c0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 09/01/2025 01:57:21.644 7f607060b7c0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:21.650 7f6076b279c0 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/01/2025 01:57:21.650 7f607060b7c0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:21.650 7f607060b7c0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:21.650 7f607060b7c0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:21.676 7f6076b279c0 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 09/01/2025 01:57:21.677 7f607060b0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:21.677 7f607060b0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 09/01/2025 01:57:21.677 7f607060b0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:21.680 7f6076b279c0 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/01/2025 01:57:21.681 7f607060b0e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:21.681 7f607060b0e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:21.681 7f607060b0e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:21.681 7f607060b900 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 09/01/2025 01:57:21.682 7f607060ba40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:21.682 7f607060ba40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 09/01/2025 01:57:21.682 7f607060ba40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:21.686 7f607060b900 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/01/2025 01:57:21.686 7f6070582d60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:21.686 7f6070582d60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:21.686 7f6070582d60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:22.025 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:23.027 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:23.355 7f6070582d60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:23.356 7f6070582d60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:23.356 7f6070582d60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:23.356 7f6070582d60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:23.356 7f6070582d60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:23.356 7f6070582d60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:23.356 7f6070582d60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:23.356 7f6070582d60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:23.357 7f6070582d60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:23.357 7f6070582d60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:23.357 7f6070582d60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:23.357 7f6070582d60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:23.357 7f6070582d60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:23.357 7f6070582d60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:24.029 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:25.030 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:26.031 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:27.033 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:27.357 7f6070582d60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:27.358 7f6070582d60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:27.358 7f6070582d60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:27.358 7f6070582d60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:27.358 7f6070582d60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:27.359 7f6070582d60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:27.359 7f6070582d60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:27.359 7f6070582d60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:27.359 7f6070582d60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:27.359 7f6070582d60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:27.359 7f6070582d60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:27.359 7f6070582d60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:27.359 7f6070582d60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:27.359 7f6070582d60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:28.034 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:29.036 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:30.037 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:31.039 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:31.359 7f6070582d60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:31.360 7f6070582d60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:31.360 7f6070582d60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:31.360 7f6070582d60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:31.360 7f6070582d60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:31.361 7f6070582d60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:31.361 7f6070582d60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:31.361 7f6070582d60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:31.361 7f6070582d60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:31.361 7f6070582d60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:31.361 7f6070582d60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:31.361 7f6070582d60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:31.361 7f6070582d60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:31.361 7f6070582d60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:32.040 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:33.042 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:34.043 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:35.045 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:35.361 7f6070582d60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:35.362 7f6070582d60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:35.362 7f6070582d60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:35.362 7f6070582d60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:35.362 7f6070582d60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:35.362 7f6070582d60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:35.362 7f6070582d60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:35.363 7f6070582d60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:35.363 7f6070582d60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:35.363 7f6070582d60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:35.363 7f6070582d60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:35.363 7f6070582d60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:35.363 7f6070582d60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:35.363 7f6070582d60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:36.047 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:37.048 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:38.050 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:39.051 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:39.363 7f6071988180 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:39.364 7f6071988180 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:39.364 7f6071988180 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:39.365 7f6071988180 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:39.365 7f6071988180 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:39.365 7f6071988180 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:39.365 7f6071988180 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:39.365 7f6071988180 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:39.365 7f6071988180 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:39.365 7f6071988180 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:39.365 7f6071988180 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:39.365 7f6071988180 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:39.366 7f6071988180 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:39.366 7f6071988180 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:40.053 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:41.055 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:42.056 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:43.058 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:43.365 7f6071988180 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:43.366 7f6071988180 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:43.367 7f6071988180 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:43.367 7f6071988180 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:43.367 7f6071988180 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:43.367 7f6071988180 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:43.367 7f6071988180 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:43.367 7f6071988180 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:43.367 7f6071988180 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:43.367 7f6071988180 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:43.367 7f6071988180 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:43.368 7f6071988180 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:43.368 7f6071988180 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:43.368 7f6071988180 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:43.470 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', '__no_rule__', 'ingress-access-control-list'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [], 'dynamic': None}, 'display_name': 'ingress-access-control-list', 'fq_name': ['default-domain', 'default-project', '__no_rule__', 'ingress-access-control-list'], 'id_perms': {'created': '2025-09-01T01:57:43.460769', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:57:43.460769', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11925350455766867329, 'uuid_mslong': 15648137040972304299}}, 'parent_type': 'security-group', 'parent_uuid': '62ca173b-4211-434e-8067-4fcb628e2505', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'd92959bb-3bc9-4fab-a57f-5b0e6a06dd81'}, 'oper': 'CREATE', 'request-id': 'req-e9e6110b-15a8-45e1-b6ff-9bbe5f1606d8', 'type': 'access_control_list', 'uuid': 'd92959bb-3bc9-4fab-a57f-5b0e6a06dd81'} 09/01/2025 01:57:43.519 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', '__no_rule__', 'egress-access-control-list'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [], 'dynamic': None}, 'display_name': 'egress-access-control-list', 'fq_name': ['default-domain', 'default-project', '__no_rule__', 'egress-access-control-list'], 'id_perms': {'created': '2025-09-01T01:57:43.512816', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:57:43.512816', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11154437056324030764, 'uuid_mslong': 14414019718996774673}}, 'parent_type': 'security-group', 'parent_uuid': '62ca173b-4211-434e-8067-4fcb628e2505', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'c808e290-d047-4f11-9acc-85676ba5d12c'}, 'oper': 'CREATE', 'request-id': 'req-a4d2e73b-68f2-4c38-a76b-9509eebec0d9', 'type': 'access_control_list', 'uuid': 'c808e290-d047-4f11-9acc-85676ba5d12c'} 09/01/2025 01:57:43.617 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['target:64512:8000000'], 'obj_dict': {'display_name': 'target:64512:8000000', 'fq_name': ['target:64512:8000000'], 'id_perms': {'created': '2025-09-01T01:57:43.610873', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:57:43.610873', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13092489608894395420, 'uuid_mslong': 18272932050593923086}}, 'perms2': {'global_access': 0, 'owner': '20dbadfeabfd403fb6d2794acee4de16', 'owner_access': 7, 'share': []}, 'uuid': 'fd967ee2-c880-400e-b5b1-ddeeaba4b41c'}, 'oper': 'CREATE', 'request-id': 'req-667bdd9c-a819-416b-a362-7c8709dec24e', 'type': 'route_target', 'uuid': 'fd967ee2-c880-400e-b5b1-ddeeaba4b41c'} 09/01/2025 01:57:43.646 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'dci-network', '__default__'], 'oper': 'UPDATE', 'request-id': 'req-d38d9b73-6e23-420e-ad61-26940b0d8417', 'type': 'routing_instance', 'uuid': '26898347-c171-4c1a-8697-64f6fe425338'} 09/01/2025 01:57:43.690 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['target:64512:8000001'], 'obj_dict': {'display_name': 'target:64512:8000001', 'fq_name': ['target:64512:8000001'], 'id_perms': {'created': '2025-09-01T01:57:43.671641', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:57:43.671641', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13570455496092117547, 'uuid_mslong': 12041940091083179522}}, 'perms2': {'global_access': 0, 'owner': '20dbadfeabfd403fb6d2794acee4de16', 'owner_access': 7, 'share': []}, 'uuid': 'a71d90b6-3f8c-4602-bc53-f15fd4d16a2b'}, 'oper': 'CREATE', 'request-id': 'req-848a89e9-1bb6-4095-a058-f8b3aa42453b', 'type': 'route_target', 'uuid': 'a71d90b6-3f8c-4602-bc53-f15fd4d16a2b'} 09/01/2025 01:57:43.713 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'dci-network', 'dci-network'], 'oper': 'UPDATE', 'request-id': 'req-7fd48cc7-6227-4e3e-9208-830d1b311a69', 'type': 'routing_instance', 'uuid': 'b402f478-6c09-4cce-978c-bf08921c9092'} 09/01/2025 01:57:43.749 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['target:64512:8000002'], 'obj_dict': {'display_name': 'target:64512:8000002', 'fq_name': ['target:64512:8000002'], 'id_perms': {'created': '2025-09-01T01:57:43.738435', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:57:43.738435', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13504741517806881909, 'uuid_mslong': 10871219233836909678}}, 'perms2': {'global_access': 0, 'owner': '20dbadfeabfd403fb6d2794acee4de16', 'owner_access': 7, 'share': []}, 'uuid': '96de544b-843e-446e-bb6a-7adbf8594c75'}, 'oper': 'CREATE', 'request-id': 'req-eafc60a5-2b5d-4bf9-8cf2-e091b0ef8565', 'type': 'route_target', 'uuid': '96de544b-843e-446e-bb6a-7adbf8594c75'} 09/01/2025 01:57:43.772 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'default-virtual-network', 'default-virtual-network'], 'oper': 'UPDATE', 'request-id': 'req-dd79017c-e717-4d14-ba03-dacf3e729740', 'type': 'routing_instance', 'uuid': 'b092f404-1958-4858-b56e-1b2fb3b6dcd2'} 09/01/2025 01:57:43.814 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['target:64512:8000003'], 'obj_dict': {'display_name': 'target:64512:8000003', 'fq_name': ['target:64512:8000003'], 'id_perms': {'created': '2025-09-01T01:57:43.808184', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:57:43.808184', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10211255544152584447, 'uuid_mslong': 14240605882422477187}}, 'perms2': {'global_access': 0, 'owner': '20dbadfeabfd403fb6d2794acee4de16', 'owner_access': 7, 'share': []}, 'uuid': 'c5a0cb99-a021-4183-8db5-aad0dbf3f0ff'}, 'oper': 'CREATE', 'request-id': 'req-e8279a5b-a4e6-4fee-aa01-864a568ee38c', 'type': 'route_target', 'uuid': 'c5a0cb99-a021-4183-8db5-aad0dbf3f0ff'} 09/01/2025 01:57:43.839 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-94fd4bc4-4c44-489d-8a70-3b0e401279f1', 'type': 'routing_instance', 'uuid': 'fa187b15-223c-4e06-9490-9a6decc1dbc7'} 09/01/2025 01:57:43.933 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [{'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'default-domain:default-project:ip-fabric'}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': None, 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'default-domain:default-project:ip-fabric'}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '00000000-0000-0000-0000-100000000001'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'deny'}, 'direction': '<>', 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'any'}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': None, 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'default-domain:default-project:ip-fabric'}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '00000000-0000-0000-0000-100000000002'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'any'}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': None, 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'any'}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '00000000-0000-0000-0000-100000000001'}], 'dynamic': False}, 'display_name': 'ip-fabric', 'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'id_perms': {'created': '2025-09-01T01:57:43.914671', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:57:43.914671', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12434635487439495936, 'uuid_mslong': 17013906886330174589}}, 'parent_type': 'virtual-network', 'parent_uuid': 'a53b9506-b472-4351-b388-6abd2a36a2e7', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'ec1d8a2b-6948-4c7d-ac90-b318244c2300'}, 'oper': 'CREATE', 'request-id': 'req-7ee2abe6-6195-4e08-b9e2-a1db250d1fde', 'type': 'access_control_list', 'uuid': 'ec1d8a2b-6948-4c7d-ac90-b318244c2300'} 09/01/2025 01:57:43.954 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', '__no_rule__', 'ingress-access-control-list'], 'oper': 'UPDATE', 'request-id': 'req-dacc4d4b-9503-4813-aead-82a4bd87ba77', 'type': 'access_control_list', 'uuid': 'd92959bb-3bc9-4fab-a57f-5b0e6a06dd81'} 09/01/2025 01:57:43.973 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', '__no_rule__', 'egress-access-control-list'], 'oper': 'UPDATE', 'request-id': 'req-28a2300c-ad5b-4055-b6ef-fd6c1340a532', 'type': 'access_control_list', 'uuid': 'c808e290-d047-4f11-9acc-85676ba5d12c'} 09/01/2025 01:57:44.059 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:45.060 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:46.062 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:47.063 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:47.367 7f6070582d60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:47.368 7f6070582d60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:47.369 7f6070582d60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:47.369 7f6070582d60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:47.370 7f6070582d60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:47.370 7f6070582d60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:47.370 7f6070582d60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:47.370 7f6070582d60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:47.370 7f6070582d60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:47.370 7f6070582d60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:47.370 7f6070582d60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:47.370 7f6070582d60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:47.370 7f6070582d60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:47.370 7f6070582d60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:47.461 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'default-global-vrouter-config'], 'oper': 'UPDATE', 'request-id': 'req-b9cee00c-1b45-404e-af54-eadf4c384e92', 'type': 'global_vrouter_config', 'uuid': '75c808b9-5f11-4499-9a9b-13d38152369a'} 09/01/2025 01:57:47.690 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'default-global-vrouter-config'], 'oper': 'UPDATE', 'request-id': 'req-bf1780e4-3d48-4dbf-97d0-5077fe0b1810', 'type': 'global_vrouter_config', 'uuid': '75c808b9-5f11-4499-9a9b-13d38152369a'} 09/01/2025 01:57:47.834 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'default-global-vrouter-config'], 'oper': 'UPDATE', 'request-id': 'req-f41da2eb-8d35-4dcf-a758-8cf2d89040a0', 'type': 'global_vrouter_config', 'uuid': '75c808b9-5f11-4499-9a9b-13d38152369a'} 09/01/2025 01:57:48.064 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:49.065 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:49.091 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-3995-1'], 'obj_dict': {'bgp_router_parameters': {'address': '10.20.0.25', 'address_families': {'family': ['route-target', 'inet-vpn', 'e-vpn', 'erm-vpn', 'inet6-vpn']}, 'admin_down': False, 'auth_data': None, 'autonomous_system': 64512, 'cluster_id': None, 'gateway_address': None, 'hold_time': 0, 'identifier': '10.20.0.25', 'ipv6_gateway_address': None, 'local_autonomous_system': None, 'port': 179, 'router_type': 'control-node', 'source_port': None, 'vendor': 'contrail'}, 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3995-1', 'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-3995-1'], 'id_perms': {'created': '2025-09-01T01:57:49.074180', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:57:49.074180', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9621979915837513096, 'uuid_mslong': 12350586173020262281}}, 'parent_type': 'routing-instance', 'parent_uuid': '24dce67f-6d24-4317-b7b6-230ab5b033b7', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'ab6618b0-2ab9-4f89-8588-23c107701d88'}, 'oper': 'CREATE', 'request-id': 'req-c0a3b8a3-b9c0-47b4-aa6d-0410935f0b7e', 'type': 'bgp_router', 'uuid': 'ab6618b0-2ab9-4f89-8588-23c107701d88'} 09/01/2025 01:57:49.365 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-bbc43391-f503-4e5f-ae26-aa04ed6e08c3', 'type': 'global_system_config', 'uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8'} 09/01/2025 01:57:49.717 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-conf-elements-inconsistency'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'ContrailConfig.elements', 'operand2': {'json_value': '1'}, 'operation': 'size=='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-conf-elements-inconsistency', 'fq_name': ['default-global-system-config', 'system-defined-conf-elements-inconsistency'], 'id_perms': {'created': '2025-09-01T01:57:49.704740', 'creator': None, 'description': 'ContrailConfig has a list of ' 'elements - it should be a dict. ' 'Controllers have inconsistency in ' 'their data.', 'enable': True, 'last_modified': '2025-09-01T01:57:49.704740', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13465058781904455200, 'uuid_mslong': 269198390335917880}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '03bc6282-7e14-4f38-badd-7f9f072b4620', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-7d26a5a0-84f7-42ec-800e-7cdc9b85f184', 'type': 'alarm', 'uuid': '03bc6282-7e14-4f38-badd-7f9f072b4620'} 09/01/2025 01:57:49.768 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-3995-3'], 'obj_dict': {'bgp_router_parameters': {'address': '10.20.0.129', 'address_families': {'family': ['route-target', 'inet-vpn', 'e-vpn', 'erm-vpn', 'inet6-vpn']}, 'admin_down': False, 'auth_data': None, 'autonomous_system': 64512, 'cluster_id': None, 'gateway_address': None, 'hold_time': 0, 'identifier': '10.20.0.129', 'ipv6_gateway_address': None, 'local_autonomous_system': None, 'port': 179, 'router_type': 'control-node', 'source_port': None, 'vendor': 'contrail'}, 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3995-3', 'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-3995-3'], 'id_perms': {'created': '2025-09-01T01:57:49.750291', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:57:49.750291', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11857703135272596087, 'uuid_mslong': 7997815229011673419}}, 'parent_type': 'routing-instance', 'parent_uuid': '24dce67f-6d24-4317-b7b6-230ab5b033b7', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '6efdf293-9873-414b-a48f-062d3df57277'}, 'oper': 'CREATE', 'request-id': 'req-8f2e7ba5-2d05-4723-9303-eb4f0aa1e337', 'type': 'bgp_router', 'uuid': '6efdf293-9873-414b-a48f-062d3df57277'} 09/01/2025 01:57:49.798 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-3995-2'], 'obj_dict': {'bgp_router_parameters': {'address': '10.20.0.193', 'address_families': {'family': ['route-target', 'inet-vpn', 'e-vpn', 'erm-vpn', 'inet6-vpn']}, 'admin_down': False, 'auth_data': None, 'autonomous_system': 64512, 'cluster_id': None, 'gateway_address': None, 'hold_time': 0, 'identifier': '10.20.0.193', 'ipv6_gateway_address': None, 'local_autonomous_system': None, 'port': 179, 'router_type': 'control-node', 'source_port': None, 'vendor': 'contrail'}, 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3995-2', 'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-3995-2'], 'id_perms': {'created': '2025-09-01T01:57:49.778205', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:57:49.778205', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10075118084394099490, 'uuid_mslong': 1863496716102945915}}, 'parent_type': 'routing-instance', 'parent_uuid': '24dce67f-6d24-4317-b7b6-230ab5b033b7', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '19dc7863-db6c-4c7b-8bd2-02847a6ca322'}, 'oper': 'CREATE', 'request-id': 'req-e5b882a7-327b-41ce-b42a-1a0c876931dc', 'type': 'bgp_router', 'uuid': '19dc7863-db6c-4c7b-8bd2-02847a6ca322'} 09/01/2025 01:57:49.839 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-conf-elements-inconsistency'], 'oper': 'UPDATE', 'request-id': 'req-02b807bd-14fd-4f1e-b82a-c25261b53d43', 'type': 'alarm', 'uuid': '03bc6282-7e14-4f38-badd-7f9f072b4620'} 09/01/2025 01:57:49.843 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-address-mismatch-compute'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'ContrailConfig.elements.virtual_router_ip_address', 'operand2': {'uve_attribute': 'VrouterAgent.control_ip'}, 'operation': '!='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-address-mismatch-compute', 'fq_name': ['default-global-system-config', 'system-defined-address-mismatch-compute'], 'id_perms': {'created': '2025-09-01T01:57:49.827148', 'creator': None, 'description': 'Compute Node IP Address mismatch.', 'enable': True, 'last_modified': '2025-09-01T01:57:49.827148', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9281412384941033885, 'uuid_mslong': 18032248181005373295}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'fa3f6a2e-8282-4f6f-80ce-33639a52499d', 'uve_keys': {'uve_key': ['vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-4d39c21d-ea49-4fbd-881e-a4e63a6130e5', 'type': 'alarm', 'uuid': 'fa3f6a2e-8282-4f6f-80ce-33639a52499d'} 09/01/2025 01:57:49.852 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-conf-elements-inconsistency'], 'oper': 'UPDATE', 'request-id': 'req-eaedbf1a-4d99-4ee5-a020-a5df73fc9289', 'type': 'alarm', 'uuid': '03bc6282-7e14-4f38-badd-7f9f072b4620'} 09/01/2025 01:57:49.889 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-3995-3'], 'oper': 'UPDATE', 'request-id': 'req-3b0d910e-4eff-4ddc-9b7a-42f2a96d0b1a', 'type': 'bgp_router', 'uuid': '6efdf293-9873-414b-a48f-062d3df57277'} 09/01/2025 01:57:49.899 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-3995-1'], 'oper': 'UPDATE-IMPLICIT', 'request-id': 'req-3b0d910e-4eff-4ddc-9b7a-42f2a96d0b1a', 'type': 'bgp_router', 'uuid': 'ab6618b0-2ab9-4f89-8588-23c107701d88'} 09/01/2025 01:57:50.012 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-3995-2'], 'oper': 'UPDATE', 'request-id': 'req-8e0e1c75-cdee-4b81-a97d-47313a4410a5', 'type': 'bgp_router', 'uuid': '19dc7863-db6c-4c7b-8bd2-02847a6ca322'} 09/01/2025 01:57:50.033 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-3995-1'], 'oper': 'UPDATE-IMPLICIT', 'request-id': 'req-8e0e1c75-cdee-4b81-a97d-47313a4410a5', 'type': 'bgp_router', 'uuid': 'ab6618b0-2ab9-4f89-8588-23c107701d88'} 09/01/2025 01:57:50.035 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-address-mismatch-control'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'ContrailConfig.elements.bgp_router_parameters.address', 'operand2': {'uve_attribute': 'BgpRouterState.bgp_router_ip_list'}, 'operation': 'not ' 'in'}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-address-mismatch-control', 'fq_name': ['default-global-system-config', 'system-defined-address-mismatch-control'], 'id_perms': {'created': '2025-09-01T01:57:49.990142', 'creator': None, 'description': 'Control Node IP Address mismatch.', 'enable': True, 'last_modified': '2025-09-01T01:57:49.990142', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13643264217144023489, 'uuid_mslong': 14422323711782898249}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'c8266300-cc83-4a49-bd56-9c8537aad9c1', 'uve_keys': {'uve_key': ['control-node']}}, 'oper': 'CREATE', 'request-id': 'req-d198a7f7-9028-417a-81b9-278fb7ce7f83', 'type': 'alarm', 'uuid': 'c8266300-cc83-4a49-bd56-9c8537aad9c1'} 09/01/2025 01:57:50.065 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:50.076 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-address-mismatch-compute'], 'oper': 'UPDATE', 'request-id': 'req-64ef5a1c-639c-4c3e-b6bf-c4d98fa5bb18', 'type': 'alarm', 'uuid': 'fa3f6a2e-8282-4f6f-80ce-33639a52499d'} 09/01/2025 01:57:50.082 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-address-mismatch-compute'], 'oper': 'UPDATE', 'request-id': 'req-6248bd5b-43c4-48ea-a023-a61144970ae3', 'type': 'alarm', 'uuid': 'fa3f6a2e-8282-4f6f-80ce-33639a52499d'} 09/01/2025 01:57:50.097 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-948d4f9f-cccb-41f5-b615-4fdb2635b0d4', 'type': 'global_system_config', 'uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8'} 09/01/2025 01:57:50.126 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-3995-2'], 'oper': 'UPDATE', 'request-id': 'req-6f660d61-332b-4f6d-adba-f26f42913c20', 'type': 'bgp_router', 'uuid': '19dc7863-db6c-4c7b-8bd2-02847a6ca322'} 09/01/2025 01:57:50.143 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-3995-3'], 'oper': 'UPDATE-IMPLICIT', 'request-id': 'req-6f660d61-332b-4f6d-adba-f26f42913c20', 'type': 'bgp_router', 'uuid': '6efdf293-9873-414b-a48f-062d3df57277'} 09/01/2025 01:57:50.144 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-45aae796-5805-4664-8f68-8e9977d1075b', 'type': 'global_system_config', 'uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8'} 09/01/2025 01:57:50.174 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-bgp-connectivity'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'BgpRouterState.num_up_bgp_peer', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}, {'and_list': [{'operand1': 'BgpRouterState.num_up_bgp_peer', 'operand2': {'uve_attribute': 'BgpRouterState.num_bgp_peer'}, 'operation': '!='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-bgp-connectivity', 'fq_name': ['default-global-system-config', 'system-defined-bgp-connectivity'], 'id_perms': {'created': '2025-09-01T01:57:50.146892', 'creator': None, 'description': 'BGP peer mismatch. Not enough BGP ' 'peers are up.', 'enable': True, 'last_modified': '2025-09-01T01:57:50.146892', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12719190076713101290, 'uuid_mslong': 9355486960643295514}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '81d55dd2-bdf1-491a-b083-a3fc512867ea', 'uve_keys': {'uve_key': ['control-node']}}, 'oper': 'CREATE', 'request-id': 'req-ab3956ca-bc75-4fc4-90b9-5fb4002eab77', 'type': 'alarm', 'uuid': '81d55dd2-bdf1-491a-b083-a3fc512867ea'} 09/01/2025 01:57:50.259 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-conf-incorrect'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'ContrailConfig', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-conf-incorrect', 'fq_name': ['default-global-system-config', 'system-defined-conf-incorrect'], 'id_perms': {'created': '2025-09-01T01:57:50.244269', 'creator': None, 'description': 'ContrailConfig missing or ' 'incorrect. Configuration pushed to ' 'Ifmap as ContrailConfig is ' 'missing/incorrect.', 'enable': True, 'last_modified': '2025-09-01T01:57:50.244269', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12345975188693128422, 'uuid_mslong': 12789451347087803108}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'b17d423d-1f4f-42e4-ab55-b705b7d714e6', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-3ebf1ad0-4d29-4eb6-b68c-83cb5e0c2c57', 'type': 'alarm', 'uuid': 'b17d423d-1f4f-42e4-ab55-b705b7d714e6'} 09/01/2025 01:57:50.279 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-address-mismatch-control'], 'oper': 'UPDATE', 'request-id': 'req-4644b680-219f-41ac-93f0-2af58d6290f6', 'type': 'alarm', 'uuid': 'c8266300-cc83-4a49-bd56-9c8537aad9c1'} 09/01/2025 01:57:50.281 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-address-mismatch-control'], 'oper': 'UPDATE', 'request-id': 'req-a1db2026-671d-43f2-beab-63ca7fd085d9', 'type': 'alarm', 'uuid': 'c8266300-cc83-4a49-bd56-9c8537aad9c1'} 09/01/2025 01:57:50.363 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-disk-usage-high'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.disk_usage_info.*.percentage_partition_space_used', 'operand2': {'json_value': '[70, ' '90]'}, 'operation': 'range', 'variables': ['NodeStatus.disk_usage_info.__key']}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-disk-usage-high', 'fq_name': ['default-global-system-config', 'system-defined-disk-usage-high'], 'id_perms': {'created': '2025-09-01T01:57:50.343229', 'creator': None, 'description': 'Disk usage crosses high threshold ' 'limit.', 'enable': True, 'last_modified': '2025-09-01T01:57:50.343229', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9410398409968651569, 'uuid_mslong': 348915920711273427}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '04d7992e-6679-4bd3-8298-737ed5150d31', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-b3df3838-b993-445d-b342-9bdc65b9c290', 'type': 'alarm', 'uuid': '04d7992e-6679-4bd3-8298-737ed5150d31'} 09/01/2025 01:57:50.452 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-bgp-connectivity'], 'oper': 'UPDATE', 'request-id': 'req-b46a4069-d0ac-41da-baab-6487258bee1f', 'type': 'alarm', 'uuid': '81d55dd2-bdf1-491a-b083-a3fc512867ea'} 09/01/2025 01:57:50.455 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-bgp-connectivity'], 'oper': 'UPDATE', 'request-id': 'req-0ea88e73-e57b-474e-b375-5282426f050e', 'type': 'alarm', 'uuid': '81d55dd2-bdf1-491a-b083-a3fc512867ea'} 09/01/2025 01:57:50.463 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-disk-usage-critical'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.disk_usage_info.*.percentage_partition_space_used', 'operand2': {'json_value': '90'}, 'operation': '>', 'variables': ['NodeStatus.disk_usage_info.__key']}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-disk-usage-critical', 'fq_name': ['default-global-system-config', 'system-defined-disk-usage-critical'], 'id_perms': {'created': '2025-09-01T01:57:50.437138', 'creator': None, 'description': 'Disk usage crosses critical ' 'threshold limit.', 'enable': True, 'last_modified': '2025-09-01T01:57:50.437138', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10407682553178176690, 'uuid_mslong': 1486187007752490893}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '149fff35-9ae3-4f8d-906f-8423cd373cb2', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-8cd87a25-7ae6-44da-ac17-b27b16f998eb', 'type': 'alarm', 'uuid': '149fff35-9ae3-4f8d-906f-8423cd373cb2'} 09/01/2025 01:57:50.553 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-node-status'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-node-status', 'fq_name': ['default-global-system-config', 'system-defined-node-status'], 'id_perms': {'created': '2025-09-01T01:57:50.537284', 'creator': None, 'description': 'Node Failure. NodeStatus UVE not ' 'present.', 'enable': True, 'last_modified': '2025-09-01T01:57:50.537284', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12650030438488176505, 'uuid_mslong': 7230954819262302855}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '64598318-cd07-4a87-af8d-efa93bf77779', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-aa98e6eb-63fe-4a1e-abf9-ce5a05ca5b01', 'type': 'alarm', 'uuid': '64598318-cd07-4a87-af8d-efa93bf77779'} 09/01/2025 01:57:50.628 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-conf-incorrect'], 'oper': 'UPDATE', 'request-id': 'req-a927b894-3265-4f46-a249-ce671ebe63da', 'type': 'alarm', 'uuid': 'b17d423d-1f4f-42e4-ab55-b705b7d714e6'} 09/01/2025 01:57:50.631 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-conf-incorrect'], 'oper': 'UPDATE', 'request-id': 'req-939925e0-8ddf-41ea-8f96-f1b4fbe1e932', 'type': 'alarm', 'uuid': 'b17d423d-1f4f-42e4-ab55-b705b7d714e6'} 09/01/2025 01:57:50.648 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-partial-sysinfo'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.build_info', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-partial-sysinfo', 'fq_name': ['default-global-system-config', 'system-defined-partial-sysinfo'], 'id_perms': {'created': '2025-09-01T01:57:50.636982', 'creator': None, 'description': 'System Info Incomplete.', 'enable': True, 'last_modified': '2025-09-01T01:57:50.636982', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11945006111573889988, 'uuid_mslong': 15521484146632968236}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'd7676398-52ea-482c-a5c5-2fc5683b53c4', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-772d7b53-1bf5-4336-9bd0-cded7ae36250', 'type': 'alarm', 'uuid': 'd7676398-52ea-482c-a5c5-2fc5683b53c4'} 09/01/2025 01:57:50.727 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-process-connectivity'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.process_status', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}, {'and_list': [{'operand1': 'NodeStatus.process_status.state', 'operand2': {'json_value': '"Functional"'}, 'operation': '!=', 'variables': ['NodeStatus.process_status.module_id', 'NodeStatus.process_status.instance_id']}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-process-connectivity', 'fq_name': ['default-global-system-config', 'system-defined-process-connectivity'], 'id_perms': {'created': '2025-09-01T01:57:50.713917', 'creator': None, 'description': 'Process(es) reporting as ' 'non-functional.', 'enable': True, 'last_modified': '2025-09-01T01:57:50.713917', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13316427422065746893, 'uuid_mslong': 7023261606246303666}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '6177a338-54f3-47b2-b8cd-742fd97fe3cd', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-17a28f51-9acd-4cc4-9846-b019f65f92ca', 'type': 'alarm', 'uuid': '6177a338-54f3-47b2-b8cd-742fd97fe3cd'} 09/01/2025 01:57:50.792 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-disk-usage-high'], 'oper': 'UPDATE', 'request-id': 'req-95b9b3ba-1a0e-4ea3-9245-25030d1a7b8c', 'type': 'alarm', 'uuid': '04d7992e-6679-4bd3-8298-737ed5150d31'} 09/01/2025 01:57:50.796 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-disk-usage-high'], 'oper': 'UPDATE', 'request-id': 'req-ff7124e2-b2c7-4010-879e-a96f7b647965', 'type': 'alarm', 'uuid': '04d7992e-6679-4bd3-8298-737ed5150d31'} 09/01/2025 01:57:50.811 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-process-status'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.process_info', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}, {'and_list': [{'operand1': 'NodeStatus.process_info.process_state', 'operand2': {'json_value': '"PROCESS_STATE_RUNNING"'}, 'operation': '!=', 'variables': ['NodeStatus.process_info.process_name']}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-process-status', 'fq_name': ['default-global-system-config', 'system-defined-process-status'], 'id_perms': {'created': '2025-09-01T01:57:50.802208', 'creator': None, 'description': 'Process Failure.', 'enable': True, 'last_modified': '2025-09-01T01:57:50.802208', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10394145945911723784, 'uuid_mslong': 9299310539720442798}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '810dc9aa-8015-43ae-903f-6caad1a28708', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-4f2c310d-2af7-42f0-b0cc-347196801623', 'type': 'alarm', 'uuid': '810dc9aa-8015-43ae-903f-6caad1a28708'} 09/01/2025 01:57:50.897 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-prouter-connectivity'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'ContrailConfig.elements.virtual_router_refs', 'operand2': {'json_value': 'null'}, 'operation': '!='}, {'operand1': 'ProuterData.connected_agent_list', 'operand2': {'json_value': '1'}, 'operation': 'size!='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-prouter-connectivity', 'fq_name': ['default-global-system-config', 'system-defined-prouter-connectivity'], 'id_perms': {'created': '2025-09-01T01:57:50.885227', 'creator': None, 'description': 'Prouter connectivity to controlling ' 'tor agent does not exist we look ' 'for non-empty value for ' 'connected_agent_list', 'enable': True, 'last_modified': '2025-09-01T01:57:50.885227', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11080958518291472377, 'uuid_mslong': 3112223694598000309}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '2b30d6f6-0211-4eb5-99c7-79101d9a1ff9', 'uve_keys': {'uve_key': ['prouter']}}, 'oper': 'CREATE', 'request-id': 'req-c2925086-dcce-45f5-ab84-4caad16fa4c3', 'type': 'alarm', 'uuid': '2b30d6f6-0211-4eb5-99c7-79101d9a1ff9'} 09/01/2025 01:57:50.970 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-disk-usage-critical'], 'oper': 'UPDATE', 'request-id': 'req-6d2502b7-dea2-4907-81a1-e4b8a763d3ee', 'type': 'alarm', 'uuid': '149fff35-9ae3-4f8d-906f-8423cd373cb2'} 09/01/2025 01:57:50.975 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-disk-usage-critical'], 'oper': 'UPDATE', 'request-id': 'req-0b329c1d-2db3-448f-97e9-07c801904437', 'type': 'alarm', 'uuid': '149fff35-9ae3-4f8d-906f-8423cd373cb2'} 09/01/2025 01:57:50.992 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-prouter-tsn-connectivity'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'ContrailConfig.elements.virtual_router_refs', 'operand2': {'json_value': 'null'}, 'operation': '!='}, {'operand1': 'ProuterData.tsn_agent_list', 'operand2': {'json_value': '1'}, 'operation': 'size!='}, {'operand1': 'ProuterData.gateway_mode', 'operand2': {'json_value': '"SERVER"'}, 'operation': '!='}]}, {'and_list': [{'operand1': 'ContrailConfig.elements.virtual_router_refs', 'operand2': {'json_value': 'null'}, 'operation': '!='}, {'operand1': 'ProuterData.tsn_agent_list', 'operand2': {'json_value': '0'}, 'operation': 'size!='}, {'operand1': 'ProuterData.gateway_mode', 'operand2': {'json_value': '"SERVER"'}, 'operation': '=='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-prouter-tsn-connectivity', 'fq_name': ['default-global-system-config', 'system-defined-prouter-tsn-connectivity'], 'id_perms': {'created': '2025-09-01T01:57:50.981229', 'creator': None, 'description': 'Prouter connectivity to controlling ' 'tsn agent does not exist we look ' 'for non-empty value for ' 'tsn_agent_list', 'enable': True, 'last_modified': '2025-09-01T01:57:50.981229', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10109986246743945171, 'uuid_mslong': 10619123826018765353}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '935eb4db-73bc-4a29-8c4d-e2ed2941efd3', 'uve_keys': {'uve_key': ['prouter']}}, 'oper': 'CREATE', 'request-id': 'req-089282b9-7dc7-43b2-a6ef-7aaed496978e', 'type': 'alarm', 'uuid': '935eb4db-73bc-4a29-8c4d-e2ed2941efd3'} 09/01/2025 01:57:51.067 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:51.074 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-storage-cluster-state'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'StorageCluster.info_stats.status', 'operand2': {'json_value': '0'}, 'operation': '!=', 'variables': ['StorageCluster.info_stats.health_summary']}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-storage-cluster-state', 'fq_name': ['default-global-system-config', 'system-defined-storage-cluster-state'], 'id_perms': {'created': '2025-09-01T01:57:51.062109', 'creator': None, 'description': 'Storage Cluster warning/errors.', 'enable': True, 'last_modified': '2025-09-01T01:57:51.062109', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11650388398251038770, 'uuid_mslong': 10237949505899020380}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '8e1480d5-277f-405c-a1ae-7e859515d832', 'uve_keys': {'uve_key': ['storage-cluster']}}, 'oper': 'CREATE', 'request-id': 'req-ddf9ec54-1868-42ef-b16e-0e94d2b51aad', 'type': 'alarm', 'uuid': '8e1480d5-277f-405c-a1ae-7e859515d832'} 09/01/2025 01:57:51.136 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-node-status'], 'oper': 'UPDATE', 'request-id': 'req-40c61117-53d3-4682-b6aa-ebab239dc2f6', 'type': 'alarm', 'uuid': '64598318-cd07-4a87-af8d-efa93bf77779'} 09/01/2025 01:57:51.141 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-node-status'], 'oper': 'UPDATE', 'request-id': 'req-372e3cc2-6865-4786-b23f-a0ea8df70b4e', 'type': 'alarm', 'uuid': '64598318-cd07-4a87-af8d-efa93bf77779'} 09/01/2025 01:57:51.157 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-vrouter-interface'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'VrouterAgent.down_interface_count', 'operand2': {'json_value': '1'}, 'operation': '>=', 'variables': ['VrouterAgent.error_intf_list', 'VrouterAgent.no_config_intf_list']}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-vrouter-interface', 'fq_name': ['default-global-system-config', 'system-defined-vrouter-interface'], 'id_perms': {'created': '2025-09-01T01:57:51.143961', 'creator': None, 'description': 'Vrouter interface(s) down.', 'enable': True, 'last_modified': '2025-09-01T01:57:51.143961', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11278860563117070971, 'uuid_mslong': 11803345335316596501}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'a3cde80b-b0cf-4715-9c86-8fec97c15a7b', 'uve_keys': {'uve_key': ['vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-1a9eb67e-f428-4a37-96a9-72466ae74d61', 'type': 'alarm', 'uuid': 'a3cde80b-b0cf-4715-9c86-8fec97c15a7b'} 09/01/2025 01:57:51.238 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-xmpp-connectivity'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'BgpRouterState.num_up_xmpp_peer', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}, {'and_list': [{'operand1': 'BgpRouterState.num_up_xmpp_peer', 'operand2': {'uve_attribute': 'BgpRouterState.num_xmpp_peer'}, 'operation': '!='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-xmpp-connectivity', 'fq_name': ['default-global-system-config', 'system-defined-xmpp-connectivity'], 'id_perms': {'created': '2025-09-01T01:57:51.226178', 'creator': None, 'description': 'XMPP peer mismatch.', 'enable': True, 'last_modified': '2025-09-01T01:57:51.226178', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11143410889604691103, 'uuid_mslong': 14282259945798913346}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'c634c7c0-0322-4d42-9aa5-5929ed1b749f', 'uve_keys': {'uve_key': ['control-node']}}, 'oper': 'CREATE', 'request-id': 'req-32967334-05f5-451f-82ec-3c34074acc51', 'type': 'alarm', 'uuid': 'c634c7c0-0322-4d42-9aa5-5929ed1b749f'} 09/01/2025 01:57:51.302 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-partial-sysinfo'], 'oper': 'UPDATE', 'request-id': 'req-b459fe8b-724f-444f-8092-ca948a7c4bb7', 'type': 'alarm', 'uuid': 'd7676398-52ea-482c-a5c5-2fc5683b53c4'} 09/01/2025 01:57:51.316 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-partial-sysinfo'], 'oper': 'UPDATE', 'request-id': 'req-c1fbc6a9-f5aa-4b42-83a9-874da7552f7a', 'type': 'alarm', 'uuid': 'd7676398-52ea-482c-a5c5-2fc5683b53c4'} 09/01/2025 01:57:51.334 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-xmpp-close-reason'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'XmppPeerInfoData.close_reason', 'operand2': {'json_value': 'null'}, 'operation': '!='}, {'operand1': 'XmppPeerInfoData.state_info.state', 'operand2': {'json_value': '"Established"'}, 'operation': '!='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-xmpp-close-reason', 'fq_name': ['default-global-system-config', 'system-defined-xmpp-close-reason'], 'id_perms': {'created': '2025-09-01T01:57:51.312606', 'creator': None, 'description': 'XMPP connection closed towards ' 'peer, ' 'alarm has close reason', 'enable': True, 'last_modified': '2025-09-01T01:57:51.312606', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13487801257560256231, 'uuid_mslong': 10235680329170243091}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '8e0c7107-46a5-4e13-bb2e-4bc845f22ee7', 'uve_keys': {'uve_key': ['xmpp-peer']}}, 'oper': 'CREATE', 'request-id': 'req-3c23439e-0ccb-4d2b-b98a-24cb37aaab0c', 'type': 'alarm', 'uuid': '8e0c7107-46a5-4e13-bb2e-4bc845f22ee7'} 09/01/2025 01:57:51.369 7f6071988180 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:51.372 7f6071988180 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:51.372 7f6071988180 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:51.373 7f6071988180 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:51.373 7f6071988180 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:51.373 7f6071988180 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:51.373 7f6071988180 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:51.373 7f6071988180 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:51.373 7f6071988180 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:51.373 7f6071988180 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:51.373 7f6071988180 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:51.374 7f6071988180 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:51.374 7f6071988180 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:51.374 7f6071988180 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:51.413 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-core-files'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.all_core_file_list', 'operand2': {'json_value': 'null'}, 'operation': '!='}, {'operand1': 'NodeStatus.all_core_file_list', 'operand2': {'json_value': '0'}, 'operation': 'size!='}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-core-files', 'fq_name': ['default-global-system-config', 'system-defined-core-files'], 'id_perms': {'created': '2025-09-01T01:57:51.398898', 'creator': None, 'description': 'A core file has been generated on ' 'the node.', 'enable': True, 'last_modified': '2025-09-01T01:57:51.398898', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10710412090359268505, 'uuid_mslong': 3024502048848363658}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '29f93096-c40c-408a-94a3-070ccbe9c499', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-da6c1f03-ff69-4dc6-a1ce-b0a85f4b848d', 'type': 'alarm', 'uuid': '29f93096-c40c-408a-94a3-070ccbe9c499'} 09/01/2025 01:57:51.478 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-process-connectivity'], 'oper': 'UPDATE', 'request-id': 'req-86a74659-a45f-42ec-840b-7a8d9d5f0e2f', 'type': 'alarm', 'uuid': '6177a338-54f3-47b2-b8cd-742fd97fe3cd'} 09/01/2025 01:57:51.486 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-process-connectivity'], 'oper': 'UPDATE', 'request-id': 'req-ff070e25-e45d-4fe3-be1d-003d05e1b568', 'type': 'alarm', 'uuid': '6177a338-54f3-47b2-b8cd-742fd97fe3cd'} 09/01/2025 01:57:51.503 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-pending-cassandra-compaction-tasks'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'CassandraStatusData.cassandra_compaction_task.pending_compaction_tasks', 'operand2': {'json_value': '300'}, 'operation': '>='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-pending-cassandra-compaction-tasks', 'fq_name': ['default-global-system-config', 'system-defined-pending-cassandra-compaction-tasks'], 'id_perms': {'created': '2025-09-01T01:57:51.495240', 'creator': None, 'description': 'Pending compaction tasks in ' 'cassandra crossed the configured ' 'threshold.', 'enable': True, 'last_modified': '2025-09-01T01:57:51.495240', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12799166691106454208, 'uuid_mslong': 15027170893856260812}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'd08b3c4f-8364-42cc-b19f-c64aec697ac0', 'uve_keys': {'uve_key': ['database-nodeconfig-database-node']}}, 'oper': 'CREATE', 'request-id': 'req-94e1d593-b6f7-40fa-a257-80aad936d295', 'type': 'alarm', 'uuid': 'd08b3c4f-8364-42cc-b19f-c64aec697ac0'} 09/01/2025 01:57:51.587 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-package-version-mismatch'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.running_package_version', 'operand2': {'uve_attribute': 'NodeStatus.installed_package_version'}, 'operation': '!='}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-package-version-mismatch', 'fq_name': ['default-global-system-config', 'system-defined-package-version-mismatch'], 'id_perms': {'created': '2025-09-01T01:57:51.577756', 'creator': None, 'description': 'There is a mismatch between ' 'installed and running package ' 'version.', 'enable': True, 'last_modified': '2025-09-01T01:57:51.577756', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10722918137435541080, 'uuid_mslong': 17809913453040846757}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'f72985ec-b0f3-47a5-94cf-753bc9704e58', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-220998bd-94b4-4899-848b-14a09e1c3183', 'type': 'alarm', 'uuid': 'f72985ec-b0f3-47a5-94cf-753bc9704e58'} 09/01/2025 01:57:51.649 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-process-status'], 'oper': 'UPDATE', 'request-id': 'req-1b0fb202-a15f-4578-a079-413e846b84b2', 'type': 'alarm', 'uuid': '810dc9aa-8015-43ae-903f-6caad1a28708'} 09/01/2025 01:57:51.658 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-process-status'], 'oper': 'UPDATE', 'request-id': 'req-5ce20252-7ee3-433a-afd4-97a83e0e1942', 'type': 'alarm', 'uuid': '810dc9aa-8015-43ae-903f-6caad1a28708'} 09/01/2025 01:57:51.676 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-vrouter-limit-exceeded'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'VrouterAgent.res_limit', 'operand2': {'json_value': 'true'}, 'operation': '==', 'variables': ['VrouterAgent.vr_limit_exceeded_list']}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-vrouter-limit-exceeded', 'fq_name': ['default-global-system-config', 'system-defined-vrouter-limit-exceeded'], 'id_perms': {'created': '2025-09-01T01:57:51.666562', 'creator': None, 'description': 'Agent resource usage exceeded ' 'configured watermark for resource.', 'enable': True, 'last_modified': '2025-09-01T01:57:51.666562', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10608329397487415044, 'uuid_mslong': 12641009152992233842}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'af6de2d9-8690-4972-9338-5b61b5922f04', 'uve_keys': {'uve_key': ['vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-043efcd0-fec7-4d8c-b54c-f5c2f0bffb7e', 'type': 'alarm', 'uuid': 'af6de2d9-8690-4972-9338-5b61b5922f04'} 09/01/2025 01:57:51.765 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-vrouter-table-limit-exceeded'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'VrouterAgent.res_table_limit', 'operand2': {'json_value': 'true'}, 'operation': '==', 'variables': ['VrouterAgent.vr_limit_exceeded_list']}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-vrouter-table-limit-exceeded', 'fq_name': ['default-global-system-config', 'system-defined-vrouter-table-limit-exceeded'], 'id_perms': {'created': '2025-09-01T01:57:51.751034', 'creator': None, 'description': 'Agent resource usage exceeded table ' 'size for resource in vrouter.', 'enable': True, 'last_modified': '2025-09-01T01:57:51.751034', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12665274423476574180, 'uuid_mslong': 14559706905692358710}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'ca0e784a-4398-4836-afc4-17fc122957e4', 'uve_keys': {'uve_key': ['vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-2b1fdcaf-0532-49a7-89d0-ff8f0b532112', 'type': 'alarm', 'uuid': 'ca0e784a-4398-4836-afc4-17fc122957e4'} 09/01/2025 01:57:51.819 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-prouter-connectivity'], 'oper': 'UPDATE', 'request-id': 'req-cceef24b-02aa-4ba9-8e77-159d86d7f996', 'type': 'alarm', 'uuid': '2b30d6f6-0211-4eb5-99c7-79101d9a1ff9'} 09/01/2025 01:57:51.830 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-prouter-connectivity'], 'oper': 'UPDATE', 'request-id': 'req-9e20dffb-8801-43ec-8834-42aff3338aa0', 'type': 'alarm', 'uuid': '2b30d6f6-0211-4eb5-99c7-79101d9a1ff9'} 09/01/2025 01:57:51.862 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-vrouter-l3mh-physical-interface-link-down'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'VrouterAgent.l3mh_phy_interface_down', 'operand2': {'json_value': 'true'}, 'operation': '==', 'variables': ['VrouterAgent.vr_l3mh_intf_list']}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-vrouter-l3mh-physical-interface-link-down', 'fq_name': ['default-global-system-config', 'system-defined-vrouter-l3mh-physical-interface-link-down'], 'id_perms': {'created': '2025-09-01T01:57:51.847772', 'creator': None, 'description': 'On l3 multihomed compute one or ' 'more physical interfaces are ' 'Inactive.', 'enable': True, 'last_modified': '2025-09-01T01:57:51.847772', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11156140797610135132, 'uuid_mslong': 437379043040970971}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '0611e1ec-1ca8-48db-9ad2-92f299cb8e5c', 'uve_keys': {'uve_key': ['vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-6e9b4c17-ad10-4551-8af6-958c601596a7', 'type': 'alarm', 'uuid': '0611e1ec-1ca8-48db-9ad2-92f299cb8e5c'} 09/01/2025 01:57:52.007 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-prouter-tsn-connectivity'], 'oper': 'UPDATE', 'request-id': 'req-0ce38428-da3d-46d3-af77-6621d179b916', 'type': 'alarm', 'uuid': '935eb4db-73bc-4a29-8c4d-e2ed2941efd3'} 09/01/2025 01:57:52.040 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-prouter-tsn-connectivity'], 'oper': 'UPDATE', 'request-id': 'req-4a3dbbdb-8ea8-4bb0-b802-e55592a31842', 'type': 'alarm', 'uuid': '935eb4db-73bc-4a29-8c4d-e2ed2941efd3'} 09/01/2025 01:57:52.069 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:52.207 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-storage-cluster-state'], 'oper': 'UPDATE', 'request-id': 'req-903126b7-3e5d-4c0d-a96c-0112009b6b6b', 'type': 'alarm', 'uuid': '8e1480d5-277f-405c-a1ae-7e859515d832'} 09/01/2025 01:57:52.226 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-storage-cluster-state'], 'oper': 'UPDATE', 'request-id': 'req-574ad662-aa66-4387-b6f3-ae5ecedfb9c0', 'type': 'alarm', 'uuid': '8e1480d5-277f-405c-a1ae-7e859515d832'} 09/01/2025 01:57:52.415 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-vrouter-interface'], 'oper': 'UPDATE', 'request-id': 'req-65485163-b46b-4b7d-b4ea-ac3e45e48b1b', 'type': 'alarm', 'uuid': 'a3cde80b-b0cf-4715-9c86-8fec97c15a7b'} 09/01/2025 01:57:52.419 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-vrouter-interface'], 'oper': 'UPDATE', 'request-id': 'req-4053dbbf-3e1e-47e0-bc3a-9121239cf4a1', 'type': 'alarm', 'uuid': 'a3cde80b-b0cf-4715-9c86-8fec97c15a7b'} 09/01/2025 01:57:52.612 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-xmpp-connectivity'], 'oper': 'UPDATE', 'request-id': 'req-b5885dd0-54b6-4de9-b985-e5729ac70731', 'type': 'alarm', 'uuid': 'c634c7c0-0322-4d42-9aa5-5929ed1b749f'} 09/01/2025 01:57:52.643 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-xmpp-connectivity'], 'oper': 'UPDATE', 'request-id': 'req-c98a9e53-e0a4-47ea-a8a2-f875f5abdca0', 'type': 'alarm', 'uuid': 'c634c7c0-0322-4d42-9aa5-5929ed1b749f'} 09/01/2025 01:57:52.796 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'default-global-vrouter-config'], 'oper': 'UPDATE', 'request-id': 'req-be77af09-c6a1-4809-9837-02e4960fbc0f', 'type': 'global_vrouter_config', 'uuid': '75c808b9-5f11-4499-9a9b-13d38152369a'} 09/01/2025 01:57:52.806 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-xmpp-close-reason'], 'oper': 'UPDATE', 'request-id': 'req-8a8b9aea-1dd9-49ef-a3fa-731e869e0879', 'type': 'alarm', 'uuid': '8e0c7107-46a5-4e13-bb2e-4bc845f22ee7'} 09/01/2025 01:57:52.815 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-xmpp-close-reason'], 'oper': 'UPDATE', 'request-id': 'req-f824c559-dd9e-47f6-b43c-74bdb695c193', 'type': 'alarm', 'uuid': '8e0c7107-46a5-4e13-bb2e-4bc845f22ee7'} 09/01/2025 01:57:52.982 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-core-files'], 'oper': 'UPDATE', 'request-id': 'req-5978a90a-a2bb-4e72-b492-04055837bab6', 'type': 'alarm', 'uuid': '29f93096-c40c-408a-94a3-070ccbe9c499'} 09/01/2025 01:57:52.985 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-core-files'], 'oper': 'UPDATE', 'request-id': 'req-cc861575-b1f8-4b01-8968-321ce8a165f7', 'type': 'alarm', 'uuid': '29f93096-c40c-408a-94a3-070ccbe9c499'} 09/01/2025 01:57:53.070 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:53.148 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-pending-cassandra-compaction-tasks'], 'oper': 'UPDATE', 'request-id': 'req-8fd3efd9-9811-4d74-9c37-1d6a2f9d9316', 'type': 'alarm', 'uuid': 'd08b3c4f-8364-42cc-b19f-c64aec697ac0'} 09/01/2025 01:57:53.164 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-pending-cassandra-compaction-tasks'], 'oper': 'UPDATE', 'request-id': 'req-d9273f8f-b94b-4a39-bdac-d445fbe24224', 'type': 'alarm', 'uuid': 'd08b3c4f-8364-42cc-b19f-c64aec697ac0'} 09/01/2025 01:57:53.337 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-package-version-mismatch'], 'oper': 'UPDATE', 'request-id': 'req-ea9f552e-98eb-4ef4-80bc-5642d6297bf3', 'type': 'alarm', 'uuid': 'f72985ec-b0f3-47a5-94cf-753bc9704e58'} 09/01/2025 01:57:53.344 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-package-version-mismatch'], 'oper': 'UPDATE', 'request-id': 'req-e3763da9-84a2-42ee-81cf-a733fdda7b1f', 'type': 'alarm', 'uuid': 'f72985ec-b0f3-47a5-94cf-753bc9704e58'} 09/01/2025 01:57:53.529 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-vrouter-limit-exceeded'], 'oper': 'UPDATE', 'request-id': 'req-00c0f81c-27b8-4bf7-b45d-8c60f682ff4c', 'type': 'alarm', 'uuid': 'af6de2d9-8690-4972-9338-5b61b5922f04'} 09/01/2025 01:57:53.532 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-vrouter-limit-exceeded'], 'oper': 'UPDATE', 'request-id': 'req-61c5b9c6-20c7-4119-a5a5-c3f28a485fa3', 'type': 'alarm', 'uuid': 'af6de2d9-8690-4972-9338-5b61b5922f04'} 09/01/2025 01:57:53.534 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'default-global-vrouter-config'], 'oper': 'UPDATE', 'request-id': 'req-fc63db98-24bf-4e4b-a4ef-4f72524f73db', 'type': 'global_vrouter_config', 'uuid': '75c808b9-5f11-4499-9a9b-13d38152369a'} 09/01/2025 01:57:53.688 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-vrouter-table-limit-exceeded'], 'oper': 'UPDATE', 'request-id': 'req-3c1aef7d-30c2-406e-983f-8730edafccde', 'type': 'alarm', 'uuid': 'ca0e784a-4398-4836-afc4-17fc122957e4'} 09/01/2025 01:57:53.695 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-vrouter-table-limit-exceeded'], 'oper': 'UPDATE', 'request-id': 'req-e4842649-b912-4285-b75d-22e7fdce5d46', 'type': 'alarm', 'uuid': 'ca0e784a-4398-4836-afc4-17fc122957e4'} 09/01/2025 01:57:53.865 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-vrouter-l3mh-physical-interface-link-down'], 'oper': 'UPDATE', 'request-id': 'req-257ec96c-0a05-47d5-8ed5-bceba5186da5', 'type': 'alarm', 'uuid': '0611e1ec-1ca8-48db-9ad2-92f299cb8e5c'} 09/01/2025 01:57:53.880 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-vrouter-l3mh-physical-interface-link-down'], 'oper': 'UPDATE', 'request-id': 'req-e087c260-1b7a-4195-aa63-9353258567c6', 'type': 'alarm', 'uuid': '0611e1ec-1ca8-48db-9ad2-92f299cb8e5c'} 09/01/2025 01:57:54.070 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:54.879 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-02593c6e-2575-40b8-a56f-63f0b1fa86fb', 'type': 'global_system_config', 'uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8'} 09/01/2025 01:57:54.969 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'default-global-vrouter-config'], 'oper': 'UPDATE', 'request-id': 'req-9a1cd17d-a4e0-4fed-a459-be82136524dd', 'type': 'global_vrouter_config', 'uuid': '75c808b9-5f11-4499-9a9b-13d38152369a'} 09/01/2025 01:57:54.977 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'default-global-vrouter-config'], 'oper': 'UPDATE', 'request-id': 'req-4438a8df-7099-48db-8441-b15938d14eed', 'type': 'global_vrouter_config', 'uuid': '75c808b9-5f11-4499-9a9b-13d38152369a'} 09/01/2025 01:57:55.072 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:55.374 7f60719b8400 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:55.375 7f60719b8400 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:55.375 7f60719b8400 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:55.375 7f60719b8400 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:55.375 7f60719b8400 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:55.375 7f60719b8400 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:55.376 7f60719b8400 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:55.376 7f60719b8400 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:55.376 7f60719b8400 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:55.376 7f60719b8400 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:55.376 7f60719b8400 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:55.376 7f60719b8400 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:55.376 7f60719b8400 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:55.376 7f60719b8400 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:56.073 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:56.230 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'default-global-vrouter-config'], 'oper': 'UPDATE', 'request-id': 'req-425b5a83-c87b-46df-8e2c-db3c3f271101', 'type': 'global_vrouter_config', 'uuid': '75c808b9-5f11-4499-9a9b-13d38152369a'} 09/01/2025 01:57:56.274 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'default-global-vrouter-config'], 'oper': 'UPDATE', 'request-id': 'req-6c0bdafe-bb88-447a-b0d8-8cbade3c0907', 'type': 'global_vrouter_config', 'uuid': '75c808b9-5f11-4499-9a9b-13d38152369a'} 09/01/2025 01:57:57.075 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:57.471 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-f00a55d7-ce9e-43e2-9395-3dd8285232ff', 'type': 'global_system_config', 'uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8'} 09/01/2025 01:57:57.533 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-3b996c4a-3334-4609-ac9b-97b4439f5e60', 'type': 'global_system_config', 'uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8'} 09/01/2025 01:57:58.076 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:59.078 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:59.376 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:59.377 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:59.377 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:59.377 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:59.378 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:59.378 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:59.378 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:59.378 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:59.378 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:59.378 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:59.378 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:59.378 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:59.379 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:59.379 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:00.080 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:01.082 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:02.083 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:03.085 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:03.378 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:03.379 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:03.379 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:03.379 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:03.379 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:03.379 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:03.379 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:03.380 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:03.380 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:03.380 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:03.380 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:03.380 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:03.380 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:03.380 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:04.086 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:05.088 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:06.090 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:07.091 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:07.380 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:07.381 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:07.382 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:07.382 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:07.382 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:07.382 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:07.382 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:07.382 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:07.382 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:07.382 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:07.383 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:07.383 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:07.383 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:07.383 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:08.093 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:09.094 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:10.096 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:11.097 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:11.382 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:11.383 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:11.383 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:11.383 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:11.384 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:11.384 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:11.384 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:11.384 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:11.384 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:11.384 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:11.384 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:11.384 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:11.384 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:11.384 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:12.099 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:13.100 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:14.102 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:15.109 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:15.385 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:15.390 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:15.390 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:15.390 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:15.390 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:15.390 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:15.390 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:15.391 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:15.392 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:15.392 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:15.392 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:15.392 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:15.392 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:15.392 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:16.111 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:17.114 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:18.115 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:19.117 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:19.391 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:19.393 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:19.393 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:19.393 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:19.393 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:19.394 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:19.394 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:19.394 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:19.394 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:19.395 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:19.395 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:19.395 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:19.395 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:19.395 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:20.119 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:20.279 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3995-2.'], 'obj_dict': {'database_node_ip_address': '10.0.0.38', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3995-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3995-2.'], 'id_perms': {'created': '2025-09-01T01:58:20.262077', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:58:20.262077', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11796017848418923458, 'uuid_mslong': 4531286052305127849}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '3ee25ca8-2354-49a9-a3b3-dfbc332d73c2'}, 'oper': 'CREATE', 'request-id': 'req-89536832-5ade-4091-a3e7-dc5b51518fac', 'type': 'database_node', 'uuid': '3ee25ca8-2354-49a9-a3b3-dfbc332d73c2'} 09/01/2025 01:58:20.290 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3995-3.'], 'obj_dict': {'database_node_ip_address': '10.0.0.131', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3995-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3995-3.'], 'id_perms': {'created': '2025-09-01T01:58:20.280218', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:58:20.280218', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10071480984782399945, 'uuid_mslong': 378361382186796020}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '054035ac-25f8-43f4-8bc5-16983c0e69c9'}, 'oper': 'CREATE', 'request-id': 'req-1e468cdd-34f7-445d-b18b-bd3cca7149c2', 'type': 'database_node', 'uuid': '054035ac-25f8-43f4-8bc5-16983c0e69c9'} 09/01/2025 01:58:20.412 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3995-1.'], 'obj_dict': {'database_node_ip_address': '10.0.0.249', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3995-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3995-1.'], 'id_perms': {'created': '2025-09-01T01:58:20.395054', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:58:20.395054', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12835641244198905252, 'uuid_mslong': 4848350493246179075}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '4348cd16-0087-4703-b221-5bb498d425a4'}, 'oper': 'CREATE', 'request-id': 'req-67f78842-b75a-435a-8bb4-1eea81bd8c1f', 'type': 'database_node', 'uuid': '4348cd16-0087-4703-b221-5bb498d425a4'} 09/01/2025 01:58:21.121 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:22.122 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:23.123 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:23.394 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:23.395 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:23.395 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:23.395 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:23.395 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:23.395 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:23.396 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:23.396 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:23.396 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:23.396 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:23.396 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:23.396 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:23.396 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:23.396 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:24.125 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:25.126 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:26.128 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:27.129 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:27.395 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:27.397 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:27.397 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:27.397 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:27.397 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:27.397 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:27.397 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:27.397 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:27.398 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:27.398 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:27.398 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:27.398 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:27.398 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:27.398 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:28.132 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:29.133 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:30.135 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:31.137 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:31.398 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:31.400 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:31.401 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:31.401 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:31.401 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:31.401 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:31.401 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:31.401 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:31.401 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:31.401 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:31.401 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:31.402 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:31.402 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:31.402 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:32.140 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:33.142 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:34.143 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:35.145 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:35.402 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:35.403 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:35.403 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:35.403 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:35.403 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:35.403 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:35.403 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:35.403 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:35.403 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:35.403 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:35.404 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:35.404 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:35.404 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:35.404 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:36.146 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:37.148 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:38.150 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:39.152 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:39.404 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:39.405 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:39.405 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:39.405 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:39.406 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:39.406 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:39.406 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:39.407 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:39.407 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:39.407 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:39.407 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:39.407 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:39.407 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:39.407 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:40.153 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:41.154 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:42.156 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:43.157 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:43.406 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:43.407 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:43.408 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:43.408 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:43.408 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:43.408 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:43.408 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:43.408 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:43.408 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:43.408 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:43.409 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:43.409 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:43.409 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:43.409 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:44.159 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:45.161 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:46.162 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:47.163 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:47.408 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:47.409 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:47.409 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:47.409 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:47.409 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:47.409 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:47.409 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:47.410 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:47.410 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:47.410 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:47.410 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:47.410 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:47.410 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:47.410 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:48.164 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:49.165 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:50.166 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:51.167 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:51.410 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:51.411 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:51.411 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:51.411 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:51.411 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:51.411 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:51.411 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:51.412 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:51.412 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:51.412 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:51.412 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:51.412 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:51.412 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:51.412 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:52.169 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:53.170 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:53.966 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3995-2.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.38', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3995-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3995-2.'], 'id_perms': {'created': '2025-09-01T01:58:53.957876', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:58:53.957876', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13676038583231394089, 'uuid_mslong': 7839128237629393148}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '6cca2d99-40a2-48fc-bdcb-0ca1f80b3129'}, 'oper': 'CREATE', 'request-id': 'req-f2c45455-00f1-4b72-b900-1a1e9bad38ff', 'type': 'analytics_node', 'uuid': '6cca2d99-40a2-48fc-bdcb-0ca1f80b3129'} 09/01/2025 01:58:53.986 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3995-3.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.131', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3995-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3995-3.'], 'id_perms': {'created': '2025-09-01T01:58:53.973832', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:58:53.973832', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10557226880738800812, 'uuid_mslong': 4644487576518083042}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '407488d7-e213-41e2-9282-cde9d7a434ac'}, 'oper': 'CREATE', 'request-id': 'req-7052381b-b338-4995-aa45-2d649573240e', 'type': 'analytics_node', 'uuid': '407488d7-e213-41e2-9282-cde9d7a434ac'} 09/01/2025 01:58:54.085 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3995-1.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.249', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3995-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3995-1.'], 'id_perms': {'created': '2025-09-01T01:58:54.061545', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:58:54.061545', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10879227890580696674, 'uuid_mslong': 10756286823668662967}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '954601e0-7afc-42b7-96fa-c8203895ae62'}, 'oper': 'CREATE', 'request-id': 'req-c09260e8-71da-48cc-a61b-d3fea836d129', 'type': 'analytics_node', 'uuid': '954601e0-7afc-42b7-96fa-c8203895ae62'} 09/01/2025 01:58:54.171 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:55.173 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:55.412 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:55.413 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:55.413 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:55.414 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:55.414 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:55.414 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:55.414 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:55.414 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:55.414 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:55.414 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:55.414 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:55.415 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:55.415 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:55.415 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:56.175 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:57.176 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:58.178 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:59.179 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:59.414 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:59.415 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:59.415 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:59.415 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:59.415 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:59.415 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:59.415 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:59.416 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:59.416 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:59.416 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:59.416 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:59.416 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:59.416 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:59.416 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:00.180 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:01.182 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:02.183 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:03.184 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:03.416 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:03.418 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:03.418 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:03.418 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:03.418 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:03.418 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:03.418 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:03.419 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:03.419 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:03.419 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:03.419 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:03.419 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:03.419 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:03.419 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:04.186 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:05.188 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:06.189 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:07.191 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:07.418 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:07.421 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:07.421 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:07.421 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:07.421 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:07.421 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:07.422 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:07.422 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:07.422 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:07.422 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:07.422 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:07.422 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:07.423 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:07.423 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:08.193 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:09.194 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:09.719 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3995-2.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.38', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3995-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3995-2.'], 'id_perms': {'created': '2025-09-01T01:59:09.711039', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:59:09.711039', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9410244128994570297, 'uuid_mslong': 7346777207241002013}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '65f4fef1-f189-4c1d-8297-e72d7e6d3039'}, 'oper': 'CREATE', 'request-id': 'req-65902cdc-006d-4333-bc45-edc4870831f6', 'type': 'analytics_alarm_node', 'uuid': '65f4fef1-f189-4c1d-8297-e72d7e6d3039'} 09/01/2025 01:59:10.102 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3995-3.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.131', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3995-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3995-3.'], 'id_perms': {'created': '2025-09-01T01:59:10.086800', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:59:10.086800', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10503918820682058891, 'uuid_mslong': 906375348849557938}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '0c941793-dd59-41b2-91c5-6a83e84b4c8b'}, 'oper': 'CREATE', 'request-id': 'req-5937b566-f775-43d5-a1c9-186dda6980e6', 'type': 'analytics_alarm_node', 'uuid': '0c941793-dd59-41b2-91c5-6a83e84b4c8b'} 09/01/2025 01:59:10.194 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3995-1.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.249', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3995-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3995-1.'], 'id_perms': {'created': '2025-09-01T01:59:10.172514', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:59:10.172514', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10879587110886434360, 'uuid_mslong': 11828816951027650397}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'a4286658-5f1f-4f5d-96fc-0ed5b74de638'}, 'oper': 'CREATE', 'request-id': 'req-5363ae59-4436-4596-ac85-f2751a51eb9c', 'type': 'analytics_alarm_node', 'uuid': 'a4286658-5f1f-4f5d-96fc-0ed5b74de638'} 09/01/2025 01:59:10.194 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:11.195 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:11.422 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:11.423 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:11.423 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:11.423 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:11.423 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:11.423 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:11.423 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:11.424 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:11.424 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:11.424 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:11.424 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:11.424 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:11.424 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:11.424 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:12.196 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:13.197 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:14.198 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:15.199 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:15.424 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:15.425 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:15.425 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:15.425 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:15.425 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:15.426 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:15.426 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:15.426 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:15.426 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:15.426 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:15.426 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:15.426 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:15.426 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:15.426 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:16.200 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:17.201 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:18.202 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:19.206 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:19.426 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:19.427 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:19.427 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:19.427 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:19.427 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:19.427 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:19.427 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:19.428 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:19.428 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:19.428 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:19.428 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:19.428 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:19.428 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:19.428 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:20.212 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:21.214 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:21.874 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3995-3.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.131', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3995-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3995-3.'], 'id_perms': {'created': '2025-09-01T01:59:21.867078', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:59:21.867078', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13422430401004267031, 'uuid_mslong': 671617895336398686}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '095210ea-dc52-4f5e-ba46-0d55aa436617'}, 'oper': 'CREATE', 'request-id': 'req-0bcb5258-1518-4af3-bc32-66c2afda7f6a', 'type': 'analytics_snmp_node', 'uuid': '095210ea-dc52-4f5e-ba46-0d55aa436617'} 09/01/2025 01:59:22.216 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:22.232 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3995-2.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.38', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3995-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3995-2.'], 'id_perms': {'created': '2025-09-01T01:59:22.227040', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:59:22.227040', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10043001507123339914, 'uuid_mslong': 13054484782216790713}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'b52ad8be-69c6-42b9-8b5f-e8a91ecca28a'}, 'oper': 'CREATE', 'request-id': 'req-cc52ae74-0d4d-419c-a865-1eba05a95d49', 'type': 'analytics_snmp_node', 'uuid': 'b52ad8be-69c6-42b9-8b5f-e8a91ecca28a'} 09/01/2025 01:59:22.309 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3995-1.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.249', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3995-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3995-1.'], 'id_perms': {'created': '2025-09-01T01:59:22.300301', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:59:22.300301', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10295412184530401449, 'uuid_mslong': 11790706739729089604}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'a3a1014f-585b-4444-8ee0-a6d59b29b8a9'}, 'oper': 'CREATE', 'request-id': 'req-79a7348d-975b-4e66-ac0e-d0c0e9ecf4b1', 'type': 'analytics_snmp_node', 'uuid': 'a3a1014f-585b-4444-8ee0-a6d59b29b8a9'} 09/01/2025 01:59:23.218 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:23.427 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:23.428 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:23.428 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:23.429 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:23.429 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:23.429 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:23.429 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:23.429 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:23.429 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:23.429 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:23.429 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:23.429 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:23.430 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:23.430 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:24.220 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:25.221 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:26.222 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:27.223 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:27.429 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:27.430 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:27.430 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:27.430 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:27.430 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:27.430 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:27.430 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:27.431 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:27.431 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:27.431 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:27.431 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:27.431 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:27.431 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:27.431 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:28.224 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:29.226 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:30.227 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:31.229 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:31.431 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:31.432 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:31.432 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:31.432 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:31.432 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:31.432 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:31.432 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:31.433 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:31.433 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:31.433 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:31.433 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:31.433 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:31.433 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:31.433 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:32.230 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:33.232 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:34.234 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:35.235 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:35.433 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:35.434 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:35.434 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:35.434 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:35.435 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:35.435 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:35.435 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:35.435 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:35.435 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:35.435 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:35.435 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:35.435 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:35.435 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:35.435 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:36.238 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:37.239 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:38.241 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:39.243 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:39.435 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:39.436 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:39.436 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:39.436 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:39.436 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:39.436 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:39.436 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:39.437 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:39.437 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:39.437 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:39.437 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:39.437 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:39.437 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:39.437 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:40.245 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:41.246 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:42.247 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:43.249 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:43.437 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:43.438 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:43.438 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:43.438 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:43.438 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:43.438 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:43.439 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:43.439 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:43.439 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:43.439 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:43.439 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:43.439 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:43.439 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:43.439 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:44.250 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:45.252 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:46.253 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:47.255 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:47.438 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:47.439 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:47.440 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:47.440 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:47.440 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:47.440 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:47.440 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:47.440 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:47.440 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:47.440 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:47.440 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:47.440 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:47.441 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:47.441 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:48.257 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:49.258 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:50.260 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:51.261 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:51.440 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:51.441 7f6071a16e00 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:51.441 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:51.441 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:51.442 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:51.442 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:51.442 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:51.442 7f6071a16e00 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:51.442 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:51.442 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:51.442 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:51.442 7f6071a16e00 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:51.442 7f6071a16e00 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:51.442 7f6071a16e00 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:52.264 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:53.265 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:54.265 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:55.267 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:55.442 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:55.443 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:55.443 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:55.443 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:55.444 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:55.444 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:55.444 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:55.444 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:55.444 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:55.444 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:55.444 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:55.444 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:55.444 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:55.444 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:56.269 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:57.270 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:58.271 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:59.273 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:59.444 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:59.445 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:59.445 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:59.445 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:59.445 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:59.445 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:59.446 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:59.446 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:59.446 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:59.446 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:59.446 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:59.446 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:59.446 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:59.446 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:00.274 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:01.276 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:02.278 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:03.279 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:03.446 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:03.448 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:03.448 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:03.448 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:03.448 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:03.448 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:03.449 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:03.449 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:03.449 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:03.449 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:03.449 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:03.449 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:03.449 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:03.449 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:04.282 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:05.283 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:06.285 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:07.286 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:07.449 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:07.451 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:07.451 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:07.451 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:07.451 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:07.451 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:07.451 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:07.452 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:07.452 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:07.452 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:07.452 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:07.452 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:07.452 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:07.452 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:08.288 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:09.289 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:10.291 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:11.293 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:11.452 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:11.453 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:11.453 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:11.453 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:11.454 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:11.454 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:11.454 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:11.454 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:11.454 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:11.454 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:11.454 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:11.454 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:11.454 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:11.455 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:12.295 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:13.296 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:14.298 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:15.300 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:15.454 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:15.456 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:15.456 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:15.456 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:15.456 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:15.457 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:15.457 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:15.457 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:15.457 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:15.457 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:15.458 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:15.458 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:15.458 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:15.458 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:16.302 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:17.304 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:18.306 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:19.306 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:19.457 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:19.458 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:19.458 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:19.458 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:19.458 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:19.458 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:19.458 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:19.458 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:19.459 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:19.459 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:19.459 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:19.459 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:19.459 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:19.459 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:20.307 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:21.309 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:22.311 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:23.314 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:23.458 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:23.460 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:23.460 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:23.460 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:23.460 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:23.460 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:23.460 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:23.461 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:23.461 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:23.461 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:23.461 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:23.461 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:23.461 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:23.461 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:24.315 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:25.317 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:26.318 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:27.318 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:27.460 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:27.461 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:27.461 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:27.462 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:27.462 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:27.462 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:27.462 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:27.462 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:27.462 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:27.462 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:27.462 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:27.462 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:27.462 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:27.463 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:28.321 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:29.323 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:30.324 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:31.325 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:31.462 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:31.464 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:31.464 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:31.464 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:31.464 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:31.464 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:31.464 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:31.464 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:31.464 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:31.465 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:31.465 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:31.465 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:31.465 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:31.465 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:32.327 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:33.329 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:34.330 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:35.331 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:35.464 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:35.467 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:35.467 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:35.467 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:35.467 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:35.467 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:35.467 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:35.467 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:35.468 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:35.468 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:35.468 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:35.468 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:35.468 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:35.468 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:36.332 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:37.333 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:38.335 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:39.336 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:39.467 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:39.468 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:39.469 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:39.469 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:39.469 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:39.469 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:39.469 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:39.469 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:39.469 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:39.469 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:39.470 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:39.470 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:39.470 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:39.470 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:40.337 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:41.339 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:42.340 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:43.341 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:43.469 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:43.471 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:43.471 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:43.471 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:43.472 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:43.472 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:43.472 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:43.473 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:43.473 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:43.473 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:43.473 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:43.473 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:43.473 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:43.473 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:44.344 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:45.345 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:46.346 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:47.348 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:47.472 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:47.473 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:47.473 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:47.473 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:47.474 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:47.474 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:47.474 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:47.474 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:47.474 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:47.474 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:47.474 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:47.474 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:47.474 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:47.475 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:48.350 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:49.351 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:50.352 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:51.354 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:51.474 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:51.475 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:51.475 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:51.475 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:51.475 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:51.476 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:51.476 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:51.476 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:51.476 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:51.476 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:51.476 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:51.476 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:51.476 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:51.476 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:52.355 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:53.357 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:54.359 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:55.360 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:55.476 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:55.478 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:55.478 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:55.478 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:55.478 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:55.478 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:55.478 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:55.479 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:55.479 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:55.479 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:55.479 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:55.479 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:55.479 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:55.479 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:56.362 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:57.364 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:58.366 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:59.367 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:59.478 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:59.480 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:59.480 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:59.480 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:59.480 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:59.480 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:59.480 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:59.481 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:59.481 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:59.481 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:59.481 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:59.481 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:59.481 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:59.481 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:00.370 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:01.371 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:02.372 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:03.374 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:03.480 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:03.482 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:03.482 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:03.482 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:03.482 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:03.482 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:03.482 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:03.482 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:03.482 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:03.483 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:03.483 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:03.483 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:03.483 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:03.483 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:04.375 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:05.376 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:06.377 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:07.379 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:07.482 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:07.484 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:07.484 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:07.484 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:07.485 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:07.485 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:07.485 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:07.485 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:07.485 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:07.485 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:07.485 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:07.485 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:07.485 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:07.486 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:08.381 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:09.382 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:10.383 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:10.477 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-d6447694-7a63-4e16-8ab1-f2203b502079', 'type': 'global_system_config', 'uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8'} 09/01/2025 02:01:10.604 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3995-2'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-3995-2', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3995-2'], 'id_perms': {'created': '2025-09-01T02:01:10.595372', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T02:01:10.595372', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11952340681741785153, 'uuid_mslong': 6586627194885784908}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '5b686689-4b18-454c-a5df-3e8619ab5841', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.192'}, 'oper': 'CREATE', 'request-id': 'req-681edb8a-0de7-491e-8459-bf06930acfe8', 'type': 'virtual_router', 'uuid': '5b686689-4b18-454c-a5df-3e8619ab5841'} 09/01/2025 02:01:10.942 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3995-2', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3995-2', 'vhost0'], 'id_perms': {'created': '2025-09-01T02:01:10.925113', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T02:01:10.925113', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10735908003063543833, 'uuid_mslong': 17648289949859202616}}, 'parent_type': 'virtual-router', 'parent_uuid': '5b686689-4b18-454c-a5df-3e8619ab5841', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'f4eb5234-6a74-4638-94fd-9b728fe03819', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:f4:eb:52:34:6a']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': 'a53b9506-b472-4351-b388-6abd2a36a2e7'}]}, 'oper': 'CREATE', 'request-id': 'req-1c3ccdca-0f5f-4532-9c6f-20034282c8ff', 'type': 'virtual_machine_interface', 'uuid': 'f4eb5234-6a74-4638-94fd-9b728fe03819'} 09/01/2025 02:01:10.965 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3995-2', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-1c3ccdca-0f5f-4532-9c6f-20034282c8ff', 'type': 'virtual_machine_interface', 'uuid': 'f4eb5234-6a74-4638-94fd-9b728fe03819'} 09/01/2025 02:01:11.384 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:11.478 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-8b202558-6c4f-4e38-a46f-f0bfe3ec33be', 'type': 'access_control_list', 'uuid': 'ec1d8a2b-6948-4c7d-ac90-b318244c2300'} 09/01/2025 02:01:11.485 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:11.486 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:11.487 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:11.487 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:11.487 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:11.487 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:11.487 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:11.487 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:11.487 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:11.487 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:11.488 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:11.488 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:11.488 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:11.488 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:12.386 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:13.387 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:14.389 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:15.391 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:15.487 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:15.489 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:15.489 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:15.489 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:15.489 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:15.489 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:15.489 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:15.490 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:15.490 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:15.490 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:15.490 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:15.490 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:15.490 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:15.490 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:16.392 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:17.394 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:18.395 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:19.397 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:19.490 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:19.491 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:19.492 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:19.492 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:19.492 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:19.492 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:19.492 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:19.492 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:19.492 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:19.493 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:19.493 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:19.493 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:19.493 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:19.493 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:20.399 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:21.401 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:22.403 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:23.404 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:23.492 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:23.493 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:23.493 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:23.493 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:23.493 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:23.493 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:23.493 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:23.494 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:23.494 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:23.494 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:23.494 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:23.494 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:23.494 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:23.494 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:24.406 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:25.408 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:26.409 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:27.410 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:27.493 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:27.495 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:27.495 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:27.495 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:27.495 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:27.495 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:27.495 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:27.495 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:27.495 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:27.496 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:27.496 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:27.496 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:27.496 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:27.496 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:28.411 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:29.413 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:30.415 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:31.416 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:31.495 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:31.496 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:31.496 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:31.496 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:31.497 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:31.497 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:31.497 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:31.497 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:31.497 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:31.497 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:31.497 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:31.497 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:31.497 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:31.497 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:32.418 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:33.420 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:34.422 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:35.423 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:35.497 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:35.498 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:35.498 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:35.498 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:35.498 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:35.498 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:35.498 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:35.499 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:35.499 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:35.499 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:35.499 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:35.499 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:35.499 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:35.499 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:36.425 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:37.427 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:38.428 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:39.429 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:39.499 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:39.500 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:39.500 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:39.500 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:39.500 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:39.500 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:39.501 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:39.501 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:39.501 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:39.501 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:39.501 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:39.501 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:39.501 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:39.501 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:40.431 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:41.432 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:42.434 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:43.435 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:43.501 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:43.502 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:43.502 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:43.502 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:43.503 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:43.503 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:43.503 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:43.503 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:43.503 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:43.503 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:43.503 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:43.503 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:43.503 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:43.503 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:44.438 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:45.439 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:46.441 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:47.442 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:47.503 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:47.504 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:47.504 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:47.504 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:47.504 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:47.504 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:47.504 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:47.505 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:47.505 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:47.505 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:47.505 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:47.505 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:47.505 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:47.505 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:48.444 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:49.446 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:50.447 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:51.449 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:51.504 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:51.506 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:51.506 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:51.506 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:51.506 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:51.506 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:51.506 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:51.507 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:51.507 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:51.507 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:51.507 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:51.507 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:51.507 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:51.507 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:52.450 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:53.450 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:54.452 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:55.453 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:55.507 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:55.508 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:55.508 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:55.508 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:55.508 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:55.508 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:55.508 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:55.509 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:55.509 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:55.509 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:55.509 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:55.509 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:55.509 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:55.509 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:56.455 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:57.457 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:58.458 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:59.460 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:59.508 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:59.509 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:59.509 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:59.510 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:59.510 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:59.510 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:59.510 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:59.510 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:59.510 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:59.510 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:59.510 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:59.510 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:59.510 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:59.510 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:00.462 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:01.463 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:02.465 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:02.992 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3995-1'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-3995-1', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3995-1'], 'id_perms': {'created': '2025-09-01T02:02:02.985199', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T02:02:02.985199', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10627036196304171688, 'uuid_mslong': 16146534956283939804}}, 'parent_type': 'global-system-config', 'parent_uuid': 'dee9192c-e41b-4855-8333-2236491cfcf8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'e01403fe-a6bb-4bdc-937a-d11db7edb2a8', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.18'}, 'oper': 'CREATE', 'request-id': 'req-7f54cc47-40c2-4259-85af-4adeb8e3ee0f', 'type': 'virtual_router', 'uuid': 'e01403fe-a6bb-4bdc-937a-d11db7edb2a8'} 09/01/2025 02:02:03.332 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3995-1', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3995-1', 'vhost0'], 'id_perms': {'created': '2025-09-01T02:02:03.316910', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T02:02:03.316910', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12779270811540185455, 'uuid_mslong': 492471125198063140}}, 'parent_type': 'virtual-router', 'parent_uuid': 'e01403fe-a6bb-4bdc-937a-d11db7edb2a8', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '06d59be1-0f91-4624-b159-17187a5ee16f', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:06:d5:9b:e1:0f']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': 'a53b9506-b472-4351-b388-6abd2a36a2e7'}]}, 'oper': 'CREATE', 'request-id': 'req-e0b4c5ea-eae1-45cd-9751-832bb456eea9', 'type': 'virtual_machine_interface', 'uuid': '06d59be1-0f91-4624-b159-17187a5ee16f'} 09/01/2025 02:02:03.357 7f607060b0e0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3995-1', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-e0b4c5ea-eae1-45cd-9751-832bb456eea9', 'type': 'virtual_machine_interface', 'uuid': '06d59be1-0f91-4624-b159-17187a5ee16f'} 09/01/2025 02:02:03.466 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:03.510 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:03.511 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:03.511 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:03.511 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:03.511 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:03.511 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:03.511 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:03.511 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:03.511 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:03.512 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:03.512 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:03.512 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:03.512 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:03.512 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:04.468 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:05.470 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:06.471 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:07.474 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:07.511 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:07.512 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:07.513 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:07.513 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:07.513 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:07.513 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:07.513 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:07.513 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:07.513 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:07.513 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:07.514 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:07.514 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:07.514 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:07.514 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:08.475 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:09.476 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:10.477 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:10.630 7f60705f7a40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:02:11.478 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:11.513 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:11.514 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:11.514 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:11.514 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:11.514 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:11.514 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:11.514 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:11.515 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:11.515 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:11.515 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:11.515 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:11.515 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:11.515 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:11.515 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:12.480 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:13.481 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:14.482 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:15.483 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:15.515 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:15.516 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:15.516 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:15.516 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:15.517 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:15.517 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:15.517 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:15.517 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:15.517 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:15.517 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:15.517 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:15.517 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:15.517 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:15.517 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:16.486 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:17.488 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:18.488 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:19.490 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:19.517 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:19.519 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:19.519 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:19.519 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:19.520 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:19.520 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:19.520 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:19.520 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:19.520 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:19.520 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:19.520 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:19.521 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:19.521 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:19.521 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:20.491 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:21.492 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:22.502 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:22.863 7f60705f7a40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:02:23.503 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:23.520 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:23.521 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:23.521 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:23.521 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:23.521 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:23.521 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:23.521 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:23.522 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:23.522 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:23.522 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:23.522 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:23.522 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:23.522 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:23.522 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:24.504 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:25.506 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:26.507 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:27.508 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:27.522 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:27.523 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:27.523 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:27.523 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:27.524 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:27.524 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:27.524 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:27.524 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:27.524 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:27.524 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:27.524 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:27.524 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:27.524 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:27.524 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:28.509 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:29.511 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:30.511 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:31.512 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:31.524 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:31.525 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:31.525 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:31.526 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:31.526 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:31.526 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:31.526 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:31.526 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:31.526 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:31.526 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:31.526 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:31.526 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:31.527 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:31.527 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:32.514 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:33.515 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:34.517 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:35.120 7f60705f7a40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:02:35.517 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:35.526 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:35.527 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:35.527 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:35.527 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:35.527 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:35.527 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:35.527 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:35.527 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:35.527 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:35.527 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:35.527 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:35.528 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:35.528 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:35.528 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:36.518 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:37.520 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:38.521 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:39.522 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:39.527 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:39.528 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:39.529 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:39.529 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:39.529 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:39.529 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:39.529 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:39.529 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:39.529 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:39.529 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:39.530 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:39.530 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:39.530 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:39.530 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:40.524 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:41.526 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:42.527 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:43.528 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:43.529 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:43.530 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:43.530 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:43.530 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:43.531 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:43.531 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:43.531 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:43.531 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:43.531 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:43.531 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:43.531 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:43.531 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:43.531 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:43.531 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:44.530 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:45.531 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:46.533 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:47.192 7f60705f7a40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:02:47.532 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:47.533 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:47.533 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:47.533 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:47.533 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:47.533 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:47.533 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:47.533 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:47.533 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:47.534 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:47.534 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:47.534 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:47.534 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:47.534 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:47.534 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:48.535 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:49.536 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:50.538 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:51.533 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:51.535 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:51.535 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:51.535 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:51.535 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:51.535 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:51.535 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:51.535 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:51.535 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:51.535 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:51.536 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:51.536 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:51.536 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:51.536 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:51.539 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:52.540 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:53.542 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:54.543 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:55.536 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:55.537 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:55.538 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:55.538 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:55.538 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:55.538 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:55.538 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:55.538 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:55.538 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:55.538 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:55.538 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:55.539 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:55.539 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:55.539 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:55.545 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:56.546 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:57.546 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:58.548 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:59.195 7f60705f7a40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:02:59.539 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:59.540 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:59.540 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:59.540 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:59.541 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:59.541 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:59.541 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:59.541 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:59.541 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:59.541 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:59.542 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:59.542 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:59.542 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:59.542 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:59.549 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:00.550 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:01.552 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:02.553 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:03.541 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:03.543 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:03.543 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:03.543 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:03.543 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:03.543 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:03.544 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:03.544 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:03.544 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:03.544 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:03.544 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:03.544 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:03.544 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:03.544 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:03.555 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:04.557 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:05.558 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:06.559 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:07.545 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:07.546 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:07.547 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:07.547 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:07.547 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:07.547 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:07.547 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:07.547 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:07.547 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:07.547 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:07.547 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:07.547 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:07.548 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:07.548 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:07.560 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:08.561 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:09.562 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:10.564 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:11.479 7f60705f7a40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:03:11.547 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:11.548 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:11.548 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:11.548 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:11.549 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:11.549 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:11.549 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:11.549 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:11.549 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:11.549 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:11.549 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:11.549 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:11.550 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:11.550 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:11.565 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:12.566 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:13.568 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:14.570 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:15.548 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:15.550 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:15.550 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:15.550 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:15.550 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:15.550 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:15.550 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:15.550 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:15.550 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:15.550 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:15.551 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:15.551 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:15.551 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:15.551 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:15.571 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:16.573 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:17.575 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:18.576 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:19.551 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:19.552 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:19.553 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:19.553 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:19.553 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:19.553 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:19.553 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:19.553 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:19.553 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:19.553 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:19.554 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:19.554 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:19.554 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:19.554 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:19.577 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:20.579 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:21.580 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:22.582 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:23.534 7f60705f7a40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:03:23.553 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:23.554 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:23.554 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:23.554 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:23.554 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:23.554 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:23.554 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:23.554 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:23.554 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:23.555 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:23.555 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:23.555 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:23.555 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:23.555 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:23.583 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:24.584 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:25.585 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:26.587 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:27.554 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:27.555 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:27.555 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:27.555 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:27.556 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:27.556 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:27.556 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:27.556 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:27.556 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:27.556 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:27.556 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:27.556 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:27.556 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:27.556 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:27.588 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:28.589 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:29.591 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:30.592 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:31.556 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:31.558 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:31.558 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:31.558 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:31.558 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:31.558 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:31.558 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:31.558 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:31.558 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:31.559 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:31.559 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:31.559 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:31.559 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:31.559 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:31.593 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:32.595 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:33.596 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:34.597 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:35.558 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:35.559 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:35.559 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:35.559 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:35.560 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:35.560 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:35.560 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:35.560 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:35.560 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:35.560 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:35.560 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:35.560 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:35.560 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:35.561 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:35.599 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:35.753 7f60705f7a40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:03:36.600 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:37.601 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:38.603 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:39.560 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:39.562 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:39.562 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:39.562 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:39.562 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:39.562 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:39.562 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:39.562 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:39.562 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:39.563 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:39.563 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:39.563 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:39.563 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:39.563 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:39.604 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:40.605 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:41.607 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:42.607 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:43.563 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:43.564 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:43.564 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:43.564 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:43.564 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:43.564 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:43.564 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:43.565 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:43.565 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:43.565 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:43.565 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:43.565 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:43.565 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:43.565 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:43.608 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:44.610 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:45.611 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:46.613 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:47.565 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:47.567 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:47.567 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:47.567 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:47.567 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:47.567 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:47.568 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:47.568 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:47.568 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:47.568 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:47.568 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:47.568 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:47.569 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:47.569 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:47.614 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:47.923 7f60705f7a40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:03:48.615 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:49.617 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:50.618 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:51.567 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:51.569 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:51.569 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:51.569 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:51.569 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:51.569 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:51.569 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:51.570 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:51.570 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:51.570 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:51.570 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:51.570 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:51.570 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:51.570 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:51.619 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:52.621 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:53.622 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:54.622 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:55.569 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:55.571 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:55.571 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:55.571 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:55.571 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:55.571 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:55.571 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:55.572 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:55.572 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:55.572 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:55.572 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:55.572 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:55.572 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:55.572 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:55.623 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:56.625 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:57.626 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:58.627 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:59.572 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:59.574 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:59.574 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:59.574 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:59.574 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:59.574 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:59.575 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:59.575 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:59.575 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:59.575 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:59.575 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:59.575 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:59.575 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:59.576 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:59.628 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:59.967 7f60705f7a40 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:04:00.630 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:01.631 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:02.632 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:03.574 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:03.576 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:03.576 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:03.576 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:03.577 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:03.577 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:03.577 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:03.577 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:03.577 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:03.578 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:03.578 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:03.578 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:03.578 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:03.578 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:03.633 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:04.635 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:05.637 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:06.638 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:07.577 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:07.579 7f60705f7a40 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:07.579 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:07.579 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:07.579 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:07.579 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:07.579 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:07.580 7f60705f7a40 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:07.580 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:07.580 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:07.580 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:07.580 7f60705f7a40 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:07.580 7f60705f7a40 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:07.580 7f60705f7a40 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:07.639 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:08.641 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:09.642 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:10.644 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:11.580 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:11.582 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:11.582 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:11.582 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:11.582 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:11.582 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:11.582 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:11.582 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:11.583 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:11.583 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:11.583 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:11.583 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:11.583 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:11.583 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:11.645 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:12.254 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:04:12.647 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:13.648 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:14.649 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:15.582 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:15.584 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:15.584 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:15.584 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:15.584 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:15.584 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:15.584 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:15.584 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:15.584 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:15.584 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:15.584 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:15.585 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:15.585 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:15.585 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:15.650 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:16.652 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:17.653 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:18.654 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:19.585 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:19.586 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:19.587 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:19.587 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:19.587 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:19.587 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:19.587 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:19.587 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:19.587 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:19.587 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:19.587 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:19.587 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:19.588 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:19.588 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:19.656 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:20.657 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:21.658 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:22.659 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:23.588 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:23.589 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:23.589 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:23.589 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:23.589 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:23.589 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:23.589 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:23.590 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:23.590 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:23.590 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:23.590 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:23.590 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:23.590 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:23.590 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:23.661 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:24.294 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:04:24.662 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:25.664 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:26.666 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:27.589 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:27.591 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:27.591 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:27.591 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:27.591 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:27.591 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:27.591 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:27.591 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:27.591 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:27.592 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:27.592 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:27.592 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:27.592 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:27.592 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:27.667 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:28.669 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:29.670 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:30.671 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:31.592 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:31.594 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:31.594 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:31.594 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:31.594 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:31.594 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:31.594 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:31.595 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:31.595 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:31.595 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:31.595 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:31.595 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:31.595 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:31.595 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:31.672 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:32.674 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:33.675 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:34.676 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:35.594 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:35.597 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:35.600 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:35.600 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:35.600 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:35.600 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:35.600 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:35.601 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:35.601 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:35.601 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:35.601 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:35.601 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:35.601 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:35.601 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:35.677 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:36.624 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:04:36.678 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:37.680 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:38.681 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:39.596 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:39.597 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:39.598 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:39.598 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:39.598 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:39.598 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:39.598 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:39.598 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:39.598 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:39.598 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:39.598 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:39.598 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:39.599 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:39.599 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:39.683 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:40.684 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:41.686 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:42.686 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:43.598 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:43.599 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:43.600 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:43.600 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:43.600 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:43.600 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:43.600 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:43.601 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:43.601 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:43.601 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:43.601 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:43.601 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:43.601 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:43.601 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:43.688 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:44.690 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:45.691 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:46.692 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:47.600 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:47.602 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:47.602 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:47.602 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:47.602 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:47.602 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:47.602 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:47.603 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:47.603 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:47.603 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:47.603 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:47.603 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:47.603 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:47.603 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:47.693 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:48.695 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:49.032 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:04:49.697 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:50.699 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:51.602 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:51.604 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:51.604 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:51.604 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:51.604 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:51.604 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:51.604 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:51.604 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:51.604 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:51.605 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:51.605 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:51.605 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:51.605 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:51.605 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:51.700 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:52.702 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:53.704 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:54.705 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:55.606 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:55.607 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:55.608 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:55.608 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:55.608 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:55.608 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:55.608 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:55.608 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:55.608 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:55.608 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:55.608 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:55.609 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:55.609 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:55.609 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:55.706 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:56.708 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:57.709 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:58.711 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:59.609 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:59.610 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:59.610 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:59.611 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:59.611 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:59.611 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:59.611 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:59.611 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:59.611 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:59.611 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:59.611 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:59.611 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:59.611 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:59.612 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:59.712 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:00.726 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:01.098 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:05:01.727 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:02.729 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:03.611 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:03.613 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:03.613 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:03.613 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:03.613 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:03.613 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:03.613 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:03.613 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:03.613 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:03.614 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:03.614 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:03.614 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:03.614 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:03.614 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:03.729 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:04.731 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:05.732 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:06.733 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:07.614 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:07.616 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:07.616 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:07.616 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:07.616 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:07.616 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:07.616 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:07.617 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:07.617 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:07.617 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:07.617 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:07.617 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:07.617 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:07.617 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:07.734 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:08.735 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:09.736 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:10.738 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:11.618 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:11.619 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:11.619 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:11.619 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:11.620 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:11.620 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:11.620 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:11.620 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:11.620 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:11.620 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:11.621 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:11.621 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:11.621 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:11.621 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:11.739 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:12.741 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:13.742 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:13.753 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:05:14.743 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:15.620 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:15.622 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:15.622 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:15.622 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:15.622 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:15.622 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:15.622 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:15.622 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:15.622 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:15.623 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:15.623 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:15.623 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:15.623 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:15.623 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:15.744 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:16.745 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:17.747 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:18.748 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:19.623 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:19.625 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:19.626 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:19.626 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:19.626 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:19.626 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:19.627 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:19.627 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:19.627 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:19.627 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:19.627 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:19.627 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:19.627 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:19.627 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:19.749 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:20.750 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:21.751 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:22.753 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:23.625 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:23.626 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:23.626 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:23.626 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:23.627 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:23.627 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:23.627 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:23.627 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:23.627 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:23.627 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:23.627 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:23.627 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:23.627 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:23.627 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:23.755 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:24.756 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:25.757 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:25.781 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:05:26.759 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:27.627 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:27.628 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:27.628 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:27.628 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:27.628 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:27.628 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:27.628 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:27.629 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:27.629 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:27.629 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:27.629 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:27.629 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:27.629 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:27.629 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:27.761 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:28.762 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:29.764 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:30.765 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:31.629 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:31.631 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:31.631 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:31.631 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:31.631 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:31.632 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:31.632 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:31.632 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:31.632 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:31.632 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:31.632 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:31.633 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:31.633 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:31.633 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:31.766 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:32.768 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:33.769 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:34.770 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:35.632 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:35.633 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:35.633 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:35.633 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:35.633 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:35.633 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:35.633 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:35.633 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:35.633 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:35.634 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:35.634 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:35.634 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:35.634 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:35.634 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:35.771 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:36.772 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:37.774 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:38.051 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:05:38.775 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:39.633 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:39.634 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:39.634 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:39.635 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:39.635 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:39.635 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:39.635 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:39.635 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:39.635 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:39.635 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:39.635 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:39.635 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:39.635 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:39.636 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:39.777 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:40.778 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:41.780 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:42.781 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:43.636 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:43.638 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:43.638 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:43.638 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:43.638 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:43.638 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:43.638 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:43.638 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:43.639 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:43.639 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:43.639 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:43.639 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:43.639 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:43.639 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:43.781 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:44.783 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:45.784 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:46.785 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:47.640 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:47.641 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:47.641 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:47.641 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:47.641 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:47.641 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:47.641 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:47.642 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:47.642 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:47.642 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:47.642 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:47.642 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:47.642 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:47.642 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:47.787 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:48.788 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:49.790 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:50.290 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:05:50.792 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:51.641 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:51.643 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:51.643 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:51.643 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:51.643 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:51.643 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:51.643 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:51.644 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:51.644 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:51.644 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:51.644 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:51.644 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:51.644 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:51.644 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:51.793 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:52.794 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:53.796 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:54.797 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:55.644 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:55.645 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:55.645 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:55.645 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:55.645 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:55.645 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:55.646 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:55.646 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:55.646 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:55.646 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:55.646 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:55.646 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:55.646 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:55.646 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:55.799 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:56.800 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:57.802 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:58.804 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:59.647 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:59.648 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:59.649 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:59.650 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:59.650 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:59.650 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:59.650 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:59.650 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:59.650 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:59.650 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:59.650 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:59.650 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:59.651 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:59.651 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:59.805 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:00.806 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:01.808 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:02.620 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:06:02.809 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:03.650 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:03.651 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:03.651 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:03.652 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:03.652 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:03.652 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:03.652 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:03.652 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:03.652 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:03.652 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:03.652 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:03.653 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:03.653 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:03.653 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:03.810 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:04.812 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:05.813 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:06.815 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:07.653 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:07.654 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:07.654 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:07.654 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:07.654 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:07.654 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:07.654 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:07.655 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:07.655 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:07.655 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:07.655 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:07.655 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:07.655 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:07.655 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:07.816 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:08.817 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:09.819 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:10.820 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:11.656 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:11.658 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:11.658 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:11.658 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:11.658 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:11.658 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:11.658 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:11.658 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:11.658 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:11.659 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:11.659 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:11.659 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:11.659 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:11.659 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:11.821 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:12.823 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:13.824 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:14.826 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:15.047 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:06:15.658 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:15.659 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:15.659 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:15.659 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:15.659 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:15.659 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:15.660 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:15.660 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:15.660 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:15.660 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:15.660 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:15.660 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:15.660 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:15.660 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:15.827 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:16.828 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:17.830 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:18.831 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:19.660 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:19.662 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:19.662 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:19.662 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:19.662 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:19.662 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:19.662 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:19.662 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:19.662 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:19.663 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:19.663 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:19.663 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:19.663 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:19.663 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:19.832 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:20.833 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:21.835 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:22.835 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:23.663 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:23.664 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:23.664 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:23.664 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:23.664 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:23.664 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:23.664 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:23.664 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:23.664 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:23.665 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:23.665 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:23.665 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:23.665 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:23.665 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:23.836 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:24.838 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:25.840 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:26.841 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:27.303 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:06:27.664 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:27.666 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:27.666 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:27.666 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:27.666 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:27.666 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:27.666 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:27.666 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:27.667 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:27.667 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:27.667 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:27.667 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:27.667 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:27.667 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:27.842 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:28.844 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:29.846 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:30.847 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:31.667 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:31.668 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:31.668 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:31.669 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:31.669 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:31.669 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:31.669 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:31.669 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:31.669 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:31.669 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:31.669 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:31.669 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:31.670 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:31.670 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:31.849 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:32.851 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:33.852 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:34.853 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:35.669 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:35.670 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:35.670 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:35.670 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:35.670 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:35.670 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:35.670 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:35.671 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:35.671 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:35.671 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:35.671 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:35.671 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:35.671 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:35.671 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:35.854 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:36.856 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:37.858 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:38.859 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:39.464 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:06:39.671 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:39.672 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:39.672 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:39.672 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:39.672 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:39.672 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:39.673 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:39.673 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:39.673 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:39.673 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:39.673 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:39.673 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:39.673 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:39.673 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:39.861 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:40.862 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:41.864 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:42.864 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:43.674 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:43.675 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:43.675 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:43.675 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:43.675 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:43.675 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:43.676 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:43.676 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:43.676 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:43.676 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:43.676 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:43.676 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:43.676 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:43.676 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:43.866 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:44.867 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:45.869 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:46.870 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:47.677 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:47.678 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:47.678 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:47.678 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:47.678 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:47.678 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:47.678 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:47.679 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:47.679 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:47.679 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:47.679 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:47.679 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:47.679 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:47.679 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:47.872 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:48.873 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:49.874 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:50.875 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:51.467 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:06:51.678 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:51.680 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:51.680 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:51.680 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:51.680 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:51.680 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:51.680 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:51.680 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:51.680 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:51.680 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:51.681 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:51.681 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:51.681 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:51.681 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:51.876 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:52.878 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:53.879 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:54.880 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:55.680 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:55.682 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:55.682 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:55.682 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:55.682 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:55.682 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:55.682 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:55.682 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:55.682 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:55.683 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:55.683 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:55.683 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:55.683 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:55.683 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:55.881 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:56.883 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:57.885 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:58.887 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:59.683 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:59.684 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:59.684 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:59.684 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:59.685 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:59.685 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:59.685 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:59.685 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:59.685 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:59.685 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:59.685 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:59.685 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:59.685 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:59.686 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:59.888 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:00.890 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:01.892 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:02.893 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:03.537 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:07:03.685 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:03.695 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:03.696 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:03.696 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:03.696 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:03.697 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:03.697 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:03.697 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:03.697 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:03.697 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:03.697 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:03.697 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:03.697 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:03.697 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:03.895 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:04.896 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:05.898 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:06.899 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:07.696 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:07.697 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:07.697 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:07.697 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:07.698 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:07.698 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:07.698 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:07.698 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:07.698 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:07.698 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:07.698 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:07.698 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:07.698 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:07.698 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:07.901 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:08.902 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:09.903 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:10.905 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:11.699 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:11.701 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:11.701 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:11.701 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:11.701 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:11.702 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:11.702 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:11.702 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:11.702 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:11.702 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:11.702 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:11.702 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:11.702 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:11.702 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:11.906 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:12.907 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:13.908 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:14.910 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:15.701 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:15.703 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:15.703 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:15.703 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:15.703 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:15.703 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:15.703 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:15.703 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:15.703 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:15.704 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:15.704 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:15.704 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:15.704 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:15.704 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:15.911 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:16.126 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:07:16.913 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:17.915 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:18.916 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:19.704 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:19.705 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:19.705 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:19.705 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:19.705 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:19.706 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:19.706 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:19.706 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:19.706 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:19.706 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:19.706 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:19.706 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:19.706 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:19.706 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:19.918 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:20.919 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:21.920 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:22.922 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:23.706 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:23.707 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:23.707 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:23.707 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:23.708 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:23.708 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:23.708 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:23.708 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:23.708 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:23.708 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:23.708 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:23.708 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:23.708 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:23.708 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:23.923 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:24.925 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:25.926 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:26.928 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:27.708 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:27.710 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:27.710 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:27.710 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:27.710 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:27.710 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:27.710 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:27.710 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:27.710 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:27.711 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:27.711 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:27.711 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:27.711 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:27.711 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:27.929 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:28.257 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:07:28.931 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:29.933 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:30.934 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:31.711 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:31.713 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:31.713 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:31.713 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:31.713 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:31.713 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:31.714 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:31.714 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:31.714 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:31.715 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:31.715 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:31.715 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:31.715 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:31.715 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:31.936 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:32.938 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:33.940 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:34.941 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:35.713 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:35.715 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:35.715 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:35.715 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:35.715 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:35.715 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:35.715 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:35.716 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:35.716 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:35.716 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:35.716 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:35.716 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:35.716 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:35.716 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:35.943 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:36.943 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:37.945 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:38.946 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:39.716 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:39.717 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:39.717 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:39.717 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:39.717 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:39.717 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:39.717 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:39.718 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:39.718 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:39.718 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:39.718 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:39.718 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:39.718 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:39.718 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:39.948 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:40.275 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:07:40.949 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:41.951 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:42.953 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:43.719 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:43.721 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:43.721 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:43.721 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:43.721 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:43.721 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:43.721 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:43.722 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:43.722 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:43.722 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:43.722 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:43.722 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:43.722 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:43.722 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:43.955 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:44.956 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:45.957 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:46.959 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:47.722 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:47.723 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:47.723 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:47.723 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:47.724 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:47.724 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:47.724 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:47.724 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:47.724 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:47.724 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:47.724 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:47.724 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:47.724 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:47.725 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:47.960 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:48.962 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:49.963 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:50.965 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:51.724 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:51.726 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:51.726 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:51.726 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:51.726 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:51.726 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:51.726 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:51.726 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:51.726 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:51.727 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:51.727 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:51.727 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:51.727 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:51.727 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:51.966 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:52.619 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:07:52.968 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:53.969 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:54.971 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:55.727 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:55.728 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:55.728 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:55.728 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:55.728 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:55.728 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:55.728 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:55.729 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:55.729 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:55.729 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:55.729 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:55.729 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:55.729 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:55.729 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:55.973 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:56.974 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:57.975 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:58.977 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:59.729 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:59.730 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:59.730 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:59.730 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:59.731 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:59.731 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:59.731 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:59.731 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:59.731 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:59.731 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:59.731 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:59.731 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:59.731 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:59.731 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:59.978 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:00.979 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:01.980 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:02.982 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:03.731 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:03.733 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:03.733 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:03.733 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:03.733 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:03.733 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:03.733 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:03.733 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:03.733 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:03.734 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:03.734 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:03.734 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:03.734 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:03.734 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:03.983 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:04.746 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:08:04.985 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:05.986 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:06.987 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:07.734 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:07.735 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:07.735 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:07.735 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:07.735 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:07.735 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:07.735 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:07.736 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:07.736 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:07.736 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:07.736 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:07.736 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:07.736 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:07.736 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:07.989 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:08.990 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:09.992 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:10.994 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:11.736 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:11.737 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:11.738 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:11.738 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:11.738 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:11.738 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:11.738 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:11.739 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:11.739 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:11.739 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:11.739 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:11.739 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:11.739 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:11.739 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:11.995 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:12.996 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:13.998 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:14.999 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:15.740 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:15.741 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:15.741 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:15.741 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:15.742 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:15.742 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:15.742 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:15.742 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:15.742 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:15.742 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:15.742 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:15.742 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:15.742 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:15.742 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:16.000 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:17.002 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:17.196 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:08:18.003 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:19.005 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:19.742 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:19.743 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:19.743 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:19.743 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:19.744 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:19.744 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:19.744 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:19.744 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:19.744 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:19.744 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:19.744 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:19.744 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:19.744 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:19.744 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:20.006 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:21.007 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:22.009 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:23.010 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:23.744 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:23.746 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:23.746 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:23.746 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:23.746 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:23.746 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:23.746 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:23.747 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:23.747 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:23.747 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:23.747 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:23.747 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:23.747 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:23.747 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:24.012 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:25.014 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:26.015 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:27.017 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:27.747 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:27.748 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:27.748 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:27.748 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:27.748 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:27.748 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:27.748 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:27.748 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:27.749 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:27.749 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:27.749 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:27.749 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:27.749 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:27.749 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:28.018 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:29.019 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:29.187 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:08:30.020 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:31.022 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:31.748 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:31.750 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:31.750 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:31.750 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:31.750 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:31.750 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:31.750 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:31.750 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:31.750 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:31.751 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:31.751 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:31.751 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:31.751 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:31.751 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:32.022 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:33.024 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:34.025 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:35.027 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:35.750 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:35.751 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:35.752 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:35.752 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:35.752 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:35.752 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:35.752 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:35.752 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:35.752 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:35.752 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:35.753 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:35.753 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:35.753 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:35.753 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:36.028 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:37.030 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:38.032 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:39.033 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:39.752 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:39.754 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:39.754 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:39.754 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:39.754 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:39.754 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:39.754 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:39.754 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:39.755 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:39.755 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:39.755 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:39.755 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:39.755 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:39.755 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:40.034 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:41.035 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:41.365 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:08:42.037 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:43.039 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:43.754 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:43.756 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:43.756 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:43.756 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:43.756 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:43.756 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:43.756 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:43.757 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:43.757 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:43.757 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:43.757 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:43.757 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:43.757 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:43.757 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:44.040 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:45.041 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:46.043 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:47.044 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:47.757 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:47.758 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:47.758 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:47.759 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:47.759 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:47.759 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:47.759 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:47.759 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:47.759 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:47.759 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:47.759 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:47.759 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:47.759 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:47.760 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:48.045 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:49.047 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:50.048 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:51.050 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:51.759 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:51.760 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:51.761 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:51.761 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:51.761 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:51.761 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:51.761 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:51.761 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:51.761 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:51.761 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:51.761 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:51.761 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:51.762 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:51.762 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:52.051 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:53.053 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:53.629 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:08:54.054 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:55.055 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:55.762 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:55.763 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:55.763 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:55.763 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:55.763 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:55.763 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:55.763 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:55.763 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:55.764 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:55.764 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:55.764 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:55.764 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:55.764 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:55.764 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:56.057 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:57.058 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:58.060 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:59.061 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:59.764 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:59.765 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:59.765 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:59.765 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:59.765 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:59.765 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:59.765 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:59.765 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:59.765 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:59.766 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:59.766 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:59.766 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:59.766 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:59.766 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:00.062 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:01.064 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:02.066 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:03.067 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:03.766 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:03.767 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:03.767 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:03.767 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:03.767 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:03.767 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:03.767 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:03.768 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:03.768 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:03.768 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:03.768 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:03.768 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:03.768 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:03.768 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:04.069 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:05.070 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:05.650 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:09:06.071 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:07.073 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:07.768 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:07.769 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:07.769 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:07.770 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:07.770 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:07.770 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:07.770 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:07.770 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:07.770 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:07.770 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:07.770 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:07.770 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:07.770 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:07.771 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:08.074 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:09.076 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:10.077 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:11.079 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:11.771 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:11.772 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:11.772 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:11.772 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:11.772 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:11.772 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:11.772 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:11.772 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:11.772 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:11.773 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:11.773 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:11.773 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:11.773 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:11.773 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:12.081 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:13.083 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:14.085 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:15.086 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:15.773 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:15.774 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:15.774 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:15.774 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:15.774 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:15.774 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:15.774 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:15.775 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:15.775 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:15.775 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:15.775 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:15.775 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:15.775 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:15.775 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:16.088 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:17.089 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:17.978 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:09:18.090 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:19.091 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:19.775 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:19.776 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:19.776 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:19.777 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:19.777 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:19.777 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:19.777 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:19.777 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:19.777 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:19.777 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:19.777 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:19.777 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:19.778 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:19.778 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:20.092 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:21.094 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:22.095 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:23.097 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:23.777 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:23.778 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:23.778 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:23.778 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:23.779 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:23.779 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:23.779 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:23.779 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:23.779 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:23.779 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:23.779 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:23.779 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:23.779 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:23.779 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:24.098 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:25.099 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:26.101 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:27.102 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:27.779 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:27.780 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:27.780 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:27.780 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:27.780 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:27.780 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:27.780 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:27.780 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:27.780 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:27.781 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:27.781 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:27.781 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:27.781 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:27.781 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:28.103 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:29.106 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:30.010 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:09:30.107 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:31.109 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:31.781 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:31.782 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:31.782 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:31.782 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:31.782 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:31.783 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:31.783 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:31.783 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:31.783 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:31.783 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:31.783 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:31.783 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:31.783 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:31.783 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:32.110 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:33.112 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:34.113 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:35.114 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:35.783 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:35.784 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:35.784 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:35.784 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:35.784 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:35.785 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:35.785 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:35.785 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:35.785 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:35.785 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:35.785 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:35.785 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:35.785 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:35.785 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:36.115 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:37.117 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:38.118 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:39.119 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:39.785 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:39.786 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:39.787 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:39.787 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:39.787 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:39.787 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:39.787 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:39.787 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:39.787 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:39.788 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:39.788 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:39.788 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:39.788 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:39.788 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:40.120 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:41.122 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:42.121 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:09:42.123 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:43.125 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:43.788 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:43.789 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:43.789 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:43.789 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:43.789 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:43.789 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:43.789 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:43.790 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:43.790 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:43.790 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:43.790 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:43.790 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:43.790 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:43.790 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:44.127 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:45.128 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:46.130 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:47.132 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:47.790 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:47.792 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:47.792 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:47.792 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:47.793 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:47.793 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:47.793 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:47.793 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:47.793 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:47.793 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:47.793 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:47.793 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:47.794 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:47.794 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:48.133 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:49.135 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:50.137 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:51.139 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:51.794 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:51.796 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:51.796 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:51.796 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:51.796 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:51.796 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:51.797 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:51.797 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:51.797 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:51.797 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:51.798 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:51.798 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:51.798 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:51.798 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:52.141 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:53.142 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:54.142 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:54.358 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:09:55.144 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:55.797 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:55.798 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:55.798 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:55.798 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:55.798 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:55.798 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:55.799 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:55.799 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:55.799 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:55.799 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:55.799 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:55.799 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:55.799 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:55.799 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:56.145 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:57.146 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:58.148 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:59.150 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:59.799 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:59.800 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:59.800 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:59.800 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:59.800 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:59.800 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:59.800 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:59.801 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:59.801 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:59.801 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:59.801 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:59.801 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:59.801 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:59.801 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:00.151 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:01.153 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:02.154 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:03.156 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:03.800 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:03.802 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:03.802 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:03.802 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:03.802 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:03.802 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:03.802 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:03.802 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:03.803 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:03.803 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:03.803 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:03.803 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:03.803 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:03.803 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:04.157 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:05.159 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:06.160 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:06.615 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:10:07.163 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:07.803 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:07.804 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:07.804 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:07.804 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:07.804 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:07.804 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:07.804 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:07.805 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:07.805 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:07.805 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:07.805 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:07.805 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:07.805 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:07.805 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:08.164 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:09.166 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:10.167 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:11.169 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:11.805 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:11.806 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:11.806 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:11.806 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:11.806 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:11.806 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:11.806 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:11.806 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:11.806 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:11.807 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:11.807 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:11.807 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:11.807 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:11.807 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:12.170 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:13.172 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:14.174 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:15.175 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:15.807 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:15.808 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:15.808 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:15.808 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:15.808 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:15.809 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:15.809 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:15.809 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:15.809 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:15.809 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:15.809 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:15.809 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:15.809 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:15.809 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:16.177 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:17.178 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:18.180 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:19.134 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:10:19.180 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:19.810 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:19.811 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:19.811 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:19.811 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:19.811 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:19.812 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:19.812 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:19.812 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:19.812 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:19.812 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:19.812 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:19.812 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:19.812 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:19.812 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:20.182 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:21.184 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:22.186 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:23.187 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:23.813 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:23.814 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:23.814 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:23.814 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:23.814 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:23.814 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:23.815 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:23.815 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:23.815 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:23.815 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:23.815 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:23.815 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:23.815 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:23.815 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:24.189 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:25.190 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:26.191 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:27.193 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:27.815 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:27.816 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:27.816 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:27.816 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:27.817 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:27.817 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:27.817 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:27.817 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:27.817 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:27.817 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:27.817 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:27.817 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:27.817 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:27.818 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:28.194 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:29.196 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:30.197 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:31.199 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:31.251 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:10:31.818 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:31.820 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:31.820 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:31.820 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:31.820 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:31.820 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:31.820 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:31.821 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:31.821 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:31.821 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:31.821 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:31.821 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:31.821 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:31.821 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:32.201 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:33.202 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:34.204 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:35.206 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:35.821 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:35.822 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:35.822 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:35.822 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:35.822 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:35.822 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:35.823 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:35.823 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:35.823 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:35.823 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:35.823 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:35.823 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:35.823 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:35.824 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:36.207 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:37.209 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:38.211 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:39.211 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:39.823 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:39.824 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:39.824 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:39.824 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:39.825 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:39.825 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:39.825 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:39.825 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:39.825 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:39.825 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:39.825 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:39.825 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:39.825 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:39.825 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:40.213 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:41.215 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:42.217 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:43.218 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:43.689 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:10:43.825 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:43.826 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:43.826 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:43.827 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:43.827 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:43.827 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:43.827 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:43.827 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:43.827 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:43.827 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:43.827 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:43.827 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:43.827 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:43.828 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:44.219 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:45.221 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:46.222 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:47.224 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:47.827 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:47.828 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:47.828 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:47.828 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:47.829 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:47.829 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:47.829 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:47.829 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:47.829 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:47.829 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:47.829 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:47.829 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:47.829 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:47.829 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:48.225 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:49.227 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:50.229 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:51.230 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:51.829 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:51.830 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:51.830 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:51.830 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:51.831 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:51.831 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:51.831 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:51.831 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:51.831 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:51.831 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:51.831 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:51.831 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:51.831 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:51.831 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:52.231 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:53.233 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:54.235 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:55.237 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:55.772 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:10:55.831 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:55.833 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:55.833 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:55.833 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:55.833 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:55.833 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:55.833 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:55.834 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:55.834 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:55.834 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:55.834 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:55.834 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:55.834 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:55.834 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:56.238 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:57.239 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:58.241 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:59.243 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:59.834 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:59.835 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:59.835 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:59.835 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:59.836 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:59.836 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:59.836 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:59.836 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:59.836 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:59.836 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:59.836 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:59.836 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:59.836 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:59.836 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:00.244 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:01.246 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:02.248 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:03.250 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:03.836 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:03.838 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:03.838 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:03.838 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:03.838 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:03.838 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:03.838 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:03.838 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:03.838 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:03.838 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:03.839 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:03.839 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:03.839 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:03.839 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:04.251 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:05.252 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:06.254 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:07.256 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:07.839 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:07.841 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:07.841 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:07.841 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:07.841 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:07.841 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:07.841 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:07.842 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:07.842 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:07.842 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:07.842 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:07.842 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:07.842 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:07.842 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:08.147 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:11:08.258 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:09.259 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:10.260 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:11.261 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:11.842 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:11.842 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:11.843 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:11.843 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:11.843 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:11.843 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:11.843 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:11.843 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:11.843 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:11.843 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:11.844 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:11.844 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:11.844 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:11.844 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:12.262 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:13.264 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:14.265 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:15.267 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:15.843 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:15.844 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:15.845 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:15.845 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:15.845 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:15.845 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:15.845 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:15.845 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:15.845 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:15.845 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:15.846 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:15.846 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:15.846 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:15.846 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:16.267 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:17.268 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:18.270 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:19.271 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:19.845 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:19.847 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:19.847 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:19.847 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:19.847 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:19.847 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:19.847 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:19.848 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:19.848 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:19.848 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:19.848 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:19.848 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:19.848 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:19.848 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:20.273 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:20.563 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:11:21.274 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:22.275 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:23.277 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:23.847 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:23.848 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:23.849 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:23.849 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:23.849 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:23.849 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:23.849 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:23.849 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:23.849 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:23.849 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:23.849 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:23.850 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:23.850 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:23.850 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:24.278 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:25.280 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:26.282 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:27.283 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:27.850 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:27.851 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:27.851 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:27.851 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:27.851 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:27.851 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:27.851 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:27.852 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:27.852 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:27.852 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:27.852 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:27.852 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:27.852 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:27.852 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:28.285 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:29.287 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:30.289 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:31.291 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:31.852 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:31.853 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:31.853 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:31.853 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:31.853 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:31.853 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:31.853 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:31.854 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:31.854 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:31.854 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:31.854 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:31.854 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:31.854 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:31.854 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:32.291 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:32.821 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:11:33.293 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:34.295 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:35.297 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:35.854 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:35.855 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:35.855 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:35.855 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:35.855 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:35.855 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:35.855 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:35.856 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:35.856 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:35.856 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:35.856 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:35.856 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:35.856 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:35.856 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:36.299 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:37.301 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:38.302 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:39.302 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:39.856 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:39.857 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:39.857 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:39.857 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:39.857 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:39.857 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:39.857 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:39.857 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:39.858 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:39.858 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:39.858 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:39.858 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:39.858 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:39.858 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:40.304 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:41.305 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:42.307 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:43.309 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:43.858 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:43.859 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:43.859 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:43.859 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:43.859 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:43.859 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:43.859 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:43.860 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:43.860 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:43.860 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:43.860 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:43.860 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:43.860 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:43.860 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:44.310 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:45.025 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:11:45.311 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:46.313 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:47.314 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:47.860 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:47.862 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:47.862 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:47.862 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:47.862 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:47.862 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:47.862 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:47.863 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:47.863 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:47.863 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:47.863 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:47.863 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:47.863 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:47.863 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:48.316 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:49.317 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:50.319 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:51.320 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:51.863 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:51.864 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:51.864 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:51.864 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:51.864 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:51.864 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:51.864 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:51.865 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:51.865 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:51.865 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:51.865 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:51.865 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:51.865 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:51.865 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:52.322 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:53.323 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:54.324 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:55.326 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:55.865 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:55.866 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:55.866 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:55.866 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:55.866 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:55.866 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:55.866 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:55.867 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:55.867 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:55.867 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:55.867 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:55.867 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:55.867 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:55.867 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:56.327 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:57.039 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:11:57.328 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:58.329 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:59.331 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:59.867 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:59.868 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:59.868 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:59.868 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:59.869 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:59.869 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:59.869 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:59.869 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:59.869 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:59.869 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:59.869 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:59.869 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:59.869 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:59.870 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:00.333 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:01.335 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:02.337 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:03.339 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:03.869 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:03.871 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:03.871 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:03.871 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:03.871 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:03.871 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:03.871 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:03.872 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:03.872 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:03.872 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:03.872 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:03.872 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:03.872 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:03.872 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:04.341 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:05.343 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:06.345 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:07.347 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:07.872 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:07.873 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:07.873 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:07.873 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:07.874 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:07.874 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:07.874 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:07.874 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:07.874 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:07.874 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:07.874 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:07.874 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:07.874 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:07.874 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:08.348 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:09.350 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:09.354 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:12:10.351 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:11.352 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:11.874 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:11.875 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:11.876 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:11.876 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:11.876 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:11.876 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:11.876 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:11.876 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:11.876 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:11.876 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:11.876 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:11.877 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:11.877 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:11.877 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:12.353 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:13.355 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:14.356 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:15.358 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:15.876 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:15.877 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:15.878 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:15.878 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:15.878 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:15.878 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:15.878 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:15.878 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:15.878 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:15.878 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:15.878 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:15.878 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:15.878 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:15.879 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:16.359 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:17.361 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:18.362 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:19.365 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:19.879 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:19.880 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:19.880 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:19.880 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:19.880 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:19.881 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:19.881 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:19.881 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:19.881 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:19.881 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:19.881 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:19.881 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:19.881 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:19.881 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:20.366 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:21.368 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:21.910 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:12:22.370 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:23.371 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:23.881 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:23.882 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:23.883 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:23.883 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:23.883 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:23.883 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:23.883 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:23.883 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:23.883 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:23.883 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:23.883 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:23.883 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:23.884 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:23.884 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:24.373 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:25.374 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:26.376 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:27.377 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:27.883 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:27.884 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:27.884 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:27.885 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:27.885 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:27.885 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:27.885 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:27.885 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:27.885 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:27.885 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:27.885 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:27.885 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:27.885 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:27.886 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:28.379 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:29.380 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:30.381 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:31.383 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:31.885 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:31.887 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:31.887 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:31.887 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:31.887 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:31.887 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:31.887 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:31.887 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:31.888 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:31.888 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:31.888 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:31.888 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:31.888 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:31.888 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:32.385 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:33.386 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:33.980 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:12:34.388 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:35.390 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:35.888 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:35.890 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:35.890 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:35.890 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:35.890 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:35.890 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:35.890 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:35.890 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:35.890 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:35.891 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:35.891 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:35.891 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:35.891 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:35.891 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:36.391 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:37.393 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:38.395 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:39.397 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:39.891 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:39.892 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:39.893 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:39.893 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:39.893 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:39.893 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:39.893 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:39.893 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:39.894 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:39.894 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:39.894 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:39.894 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:39.894 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:39.894 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:40.399 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:41.401 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:42.402 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:43.404 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:43.894 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:43.895 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:43.895 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:43.896 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:43.896 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:43.896 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:43.896 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:43.896 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:43.896 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:43.896 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:43.896 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:43.896 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:43.897 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:43.897 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:44.405 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:45.407 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:46.085 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:12:46.408 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:47.410 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:47.896 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:47.898 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:47.898 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:47.899 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:47.899 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:47.899 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:47.899 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:47.899 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:47.899 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:47.899 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:47.899 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:47.899 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:47.900 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:47.900 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:48.411 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:49.413 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:50.415 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:51.417 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:51.899 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:51.901 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:51.901 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:51.901 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:51.901 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:51.901 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:51.901 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:51.902 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:51.902 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:51.902 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:51.902 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:51.902 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:51.902 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:51.902 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:52.418 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:53.420 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:54.421 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:55.423 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:55.902 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:55.903 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:55.904 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:55.904 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:55.904 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:55.904 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:55.904 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:55.904 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:55.904 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:55.904 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:55.905 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:55.905 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:55.905 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:55.905 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:56.425 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:57.426 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:58.361 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:12:58.427 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:59.429 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:59.904 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:59.906 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:59.906 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:59.906 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:59.906 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:59.906 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:59.906 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:59.907 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:59.907 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:59.907 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:59.907 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:59.907 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:59.907 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:59.907 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:00.431 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:01.433 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:02.434 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:03.435 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:03.907 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:03.908 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:03.909 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:03.909 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:03.909 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:03.909 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:03.909 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:03.909 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:03.909 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:03.910 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:03.910 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:03.910 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:03.910 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:03.910 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:04.437 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:05.439 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:06.441 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:07.443 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:07.909 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:07.911 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:07.911 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:07.911 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:07.911 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:07.911 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:07.911 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:07.912 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:07.912 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:07.912 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:07.912 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:07.912 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:07.912 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:07.912 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:08.445 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:09.447 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:10.449 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:10.588 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:13:11.451 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:11.912 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:11.913 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:11.914 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:11.914 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:11.914 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:11.914 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:11.914 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:11.914 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:11.915 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:11.915 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:11.915 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:11.915 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:11.915 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:11.915 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:12.453 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:13.455 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:14.457 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:15.457 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:15.914 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:15.916 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:15.916 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:15.916 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:15.916 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:15.916 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:15.916 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:15.916 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:15.916 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:15.917 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:15.917 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:15.917 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:15.917 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:15.917 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:16.459 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:17.460 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:18.462 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:19.464 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:19.917 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:19.918 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:19.918 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:19.918 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:19.919 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:19.919 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:19.919 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:19.919 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:19.919 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:19.919 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:19.919 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:19.919 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:19.919 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:19.920 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:20.465 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:21.467 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:22.469 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:23.148 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:13:23.471 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:23.919 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:23.920 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:23.920 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:23.920 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:23.921 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:23.921 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:23.921 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:23.921 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:23.921 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:23.921 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:23.921 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:23.921 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:23.921 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:23.921 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:24.473 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:25.475 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:26.477 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:27.478 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:27.921 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:27.922 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:27.923 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:27.923 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:27.923 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:27.923 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:27.923 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:27.923 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:27.923 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:27.924 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:27.924 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:27.924 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:27.924 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:27.924 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:28.479 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:29.481 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:30.482 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:31.484 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:31.924 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:31.926 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:31.926 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:31.926 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:31.926 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:31.926 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:31.926 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:31.926 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:31.927 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:31.927 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:31.927 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:31.927 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:31.927 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:31.927 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:32.486 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:33.488 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:34.489 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:35.335 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:13:35.490 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:35.926 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:35.928 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:35.928 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:35.928 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:35.928 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:35.928 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:35.928 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:35.929 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:35.929 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:35.929 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:35.929 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:35.929 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:35.929 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:35.929 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:36.505 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:37.507 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:38.509 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:39.511 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:39.929 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:39.930 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:39.930 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:39.930 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:39.931 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:39.931 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:39.931 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:39.931 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:39.931 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:39.931 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:39.931 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:39.931 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:39.931 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:39.931 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:40.512 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:41.514 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:42.516 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:43.517 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:43.932 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:43.933 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:43.934 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:43.934 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:43.934 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:43.934 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:43.934 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:43.934 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:43.934 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:43.935 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:43.935 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:43.935 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:43.935 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:43.935 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:44.518 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:45.520 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:46.522 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:47.523 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:47.616 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:13:47.934 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:47.935 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:47.936 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:47.936 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:47.936 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:47.936 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:47.936 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:47.936 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:47.936 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:47.936 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:47.936 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:47.937 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:47.937 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:47.937 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:48.525 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:49.527 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:50.528 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:51.530 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:51.937 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:51.938 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:51.939 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:51.939 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:51.939 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:51.939 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:51.939 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:51.939 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:51.939 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:51.940 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:51.940 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:51.940 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:51.940 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:51.940 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:52.532 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:53.534 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:54.535 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:55.537 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:55.940 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:55.942 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:55.942 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:55.942 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:55.942 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:55.942 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:55.942 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:55.942 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:55.943 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:55.943 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:55.943 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:55.943 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:55.943 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:55.943 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:56.539 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:57.541 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:58.542 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:59.544 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:59.661 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:13:59.942 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:59.943 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:59.944 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:59.944 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:59.944 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:59.944 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:59.944 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:59.944 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:59.944 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:59.944 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:59.944 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:59.944 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:59.945 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:59.945 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:00.546 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:01.547 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:02.549 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:03.550 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:03.944 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:03.945 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:03.946 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:03.946 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:03.946 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:03.946 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:03.946 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:03.946 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:03.946 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:03.946 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:03.946 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:03.946 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:03.947 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:03.947 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:04.551 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:05.553 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:06.554 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:07.556 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:07.946 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:07.947 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:07.947 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:07.947 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:07.948 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:07.948 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:07.948 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:07.948 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:07.948 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:07.948 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:07.948 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:07.948 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:07.948 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:07.948 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:08.557 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:09.558 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:10.560 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:11.561 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:11.630 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:14:11.948 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:11.949 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:11.949 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:11.949 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:11.949 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:11.949 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:11.949 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:11.950 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:11.950 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:11.950 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:11.950 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:11.950 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:11.950 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:11.950 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:12.563 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:13.565 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:14.567 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:15.568 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:15.949 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:15.951 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:15.951 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:15.951 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:15.951 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:15.951 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:15.951 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:15.952 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:15.952 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:15.952 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:15.952 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:15.952 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:15.952 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:15.952 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:16.570 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:17.572 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:18.573 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:19.575 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:19.951 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:19.953 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:19.953 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:19.953 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:19.953 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:19.953 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:19.953 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:19.953 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:19.954 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:19.954 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:19.954 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:19.954 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:19.954 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:19.954 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:20.576 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:21.577 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:22.578 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:23.579 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:23.955 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:23.956 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:23.956 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:23.956 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:23.956 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:23.956 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:23.956 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:23.957 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:23.957 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:23.957 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:23.957 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:23.957 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:23.957 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:23.957 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:24.199 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:14:24.581 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:25.583 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:26.212 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:14:26.584 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:27.586 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:27.956 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:27.957 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:27.957 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:27.958 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:27.958 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:27.958 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:27.958 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:27.958 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:27.958 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:27.958 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:27.958 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:27.958 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:27.958 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:27.959 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:28.206 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:14:28.587 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:29.588 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:30.590 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:31.591 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:31.958 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:31.959 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:31.960 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:31.960 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:31.960 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:31.960 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:31.960 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:31.960 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:31.960 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:31.961 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:31.961 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:31.961 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:31.961 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:31.961 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:32.593 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:33.594 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:34.595 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:35.596 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:35.960 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:35.961 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:35.961 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:35.961 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:35.961 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:35.961 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:35.961 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:35.962 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:35.962 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:35.962 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:35.962 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:35.962 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:35.962 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:35.962 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:36.598 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:37.599 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:38.600 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:39.602 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:39.962 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:39.964 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:39.964 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:39.964 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:39.964 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:39.964 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:39.964 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:39.965 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:39.965 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:39.965 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:39.965 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:39.965 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:39.965 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:39.966 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:40.606 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:41.607 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:42.609 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:43.611 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:43.965 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:43.966 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:43.966 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:43.966 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:43.966 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:43.966 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:43.966 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:43.967 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:43.967 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:43.967 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:43.967 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:43.967 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:43.967 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:43.967 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:44.612 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:45.613 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:45.815 7f60719b8220 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:14:46.615 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:47.616 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:47.967 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:47.968 7f60719b8220 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:47.968 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:47.968 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:47.969 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:47.969 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:47.969 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:47.969 7f60719b8220 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:47.969 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:47.969 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.131:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.249:5673, 10.0.0.38:5673, 10.0.0.131:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.249:9041, 10.0.0.38:9041, 10.0.0.131:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.249:2181, 10.0.0.38:2181, 10.0.0.131:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.249,10.0.0.38,10.0.0.131:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:47.969 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:47.969 7f60719b8220 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:47.969 7f60719b8220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3995-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 132 http_port = 8096 start_time = 1756691795326032 collector_name = collector_ip = 10.0.0.131:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.38:8086, 10.0.0.131:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:47.969 7f60719b8220 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:48.618 7f6071724540 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process.