07/07/2025 00:44:38.225 7f8cf582b2c8 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 07/07/2025 00:44:38.225 7f8cf582b2c8 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 07/07/2025 00:44:38.225 7f8cf582b2c8 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 07/07/2025 00:44:38.225 7f8cecf372e8 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 07/07/2025 00:44:38.226 7f8cecf372e8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 07/07/2025 00:44:38.227 7f8cecf372e8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 07/07/2025 00:44:38.227 7f8cecf372e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 07/07/2025 00:44:38.227 7f8cecf372e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:38.227 7f8cecf372e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 07/07/2025 00:44:38.227 7f8cecf372e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:38.227 7f8cecf372e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:38.227 7f8cecf372e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1751849078226926 collector_name = collector_ip = collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 07/07/2025 00:44:38.227 7f8cecf372e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:38.278 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.41:5673//', 'pyamqp://guest:guest@10.0.0.27:5673//', 'pyamqp://guest:guest@10.0.0.23:5673//'] 07/07/2025 00:44:38.288 7f8cf582b2c8 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 07/07/2025 00:44:38.289 7f8cece87588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:38.289 7f8cece87588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 07/07/2025 00:44:38.289 7f8cece87588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:38.289 7f8cece87588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:38.290 7f8cece87588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 07/07/2025 00:44:38.290 7f8cece87588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:40.300 7f8cf582b2c8 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 07/07/2025 00:44:40.300 7f8cecd23588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:40.301 7f8cecd23588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 07/07/2025 00:44:40.301 7f8cecd23588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:40.311 7f8cf582b2c8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 07/07/2025 00:44:40.311 7f8cecea6198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:40.312 7f8cecea6198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 07/07/2025 00:44:40.312 7f8cecea6198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:40.314 7f8cf582b2c8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 07/07/2025 00:44:40.314 7f8cfb73c978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:40.314 7f8cfb73c978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 07/07/2025 00:44:40.314 7f8cfb73c978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:40.322 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 07/07/2025 00:44:40.323 7f8cf582b2c8 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 07/07/2025 00:44:40.323 7f8cf582b2c8 [contrail-device-manager] [WARNING]: initializing cassandra cluster 07/07/2025 00:44:40.323 7f8cf582b2c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 07/07/2025 00:44:40.323 7f8cf58a1f60 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 07/07/2025 00:44:40.323 7f8cf58a1f60 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 07/07/2025 00:44:40.324 7f8cece87240 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 07/07/2025 00:44:40.324 7f8cece87240 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 07/07/2025 00:44:40.324 7f8cecd23eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:40.324 7f8cecd23eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 07/07/2025 00:44:40.324 7f8cecd23eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:40.325 7f8cf582b2c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 07/07/2025 00:44:40.328 7f8cece87240 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 07/07/2025 00:44:40.329 7f8cf58a1f60 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 07/07/2025 00:44:40.329 7f8cf58a1f60 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 07/07/2025 00:44:40.358 7f8cf582b2c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 07/07/2025 00:44:40.360 7f8cecd02198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:40.360 7f8cecd02198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 07/07/2025 00:44:40.360 7f8cecd02198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:40.366 7f8cf582b2c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 07/07/2025 00:44:40.366 7f8ceccfd048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:40.367 7f8ceccfd048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 07/07/2025 00:44:40.367 7f8ceccfd048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:40.373 7f8cf582b2c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 07/07/2025 00:44:40.398 7f8cf582b2c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 07/07/2025 00:44:41.983 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/07/2025 00:44:42.227 7f8cecc702e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:44:42.228 7f8cecc702e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:44:42.228 7f8cecc702e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:44:42.228 7f8cecc702e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:44:42.229 7f8cecc702e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 07/07/2025 00:44:42.229 7f8cecc702e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:44:42.229 7f8cecc702e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:44:42.229 7f8cecc702e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:44:42.229 7f8cecc702e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:42.229 7f8cecc702e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 07/07/2025 00:44:42.229 7f8cecc702e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:42.229 7f8cecc702e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:42.229 7f8cecc702e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.27:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:44:42.229 7f8cecc702e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:43.138 7f8cf582b2c8 [contrail-device-manager] [WARNING]: table 'obj_uuid_table', created 07/07/2025 00:44:43.143 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/07/2025 00:44:45.162 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/07/2025 00:44:46.055 7f8cf582b2c8 [contrail-device-manager] [WARNING]: table 'obj_uuid_table' fixed with properties {'gc_grace_seconds': 864000} 07/07/2025 00:44:46.060 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/07/2025 00:44:46.229 7f8cecc706d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:44:46.230 7f8cecc706d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:44:46.230 7f8cecc706d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:44:46.230 7f8cecc706d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:44:46.230 7f8cecc706d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 07/07/2025 00:44:46.230 7f8cecc706d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:44:46.231 7f8cecc706d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:44:46.231 7f8cecc706d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:44:46.231 7f8cecc706d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:46.231 7f8cecc706d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 07/07/2025 00:44:46.231 7f8cecc706d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:46.231 7f8cecc706d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:46.231 7f8cecc706d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:44:46.231 7f8cecc706d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:48.282 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/07/2025 00:44:48.285 7f8cf582b2c8 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table', created 07/07/2025 00:44:48.290 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/07/2025 00:44:50.124 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/07/2025 00:44:50.230 7f8cec42ff60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:44:50.232 7f8cec42ff60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:44:50.232 7f8cec42ff60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:44:50.232 7f8cec42ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:44:50.232 7f8cec42ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 07/07/2025 00:44:50.232 7f8cec42ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:44:50.232 7f8cec42ff60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:44:50.232 7f8cec42ff60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:44:50.233 7f8cec42ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:50.233 7f8cec42ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 07/07/2025 00:44:50.233 7f8cec42ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:50.233 7f8cec42ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:50.233 7f8cec42ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:44:50.233 7f8cec42ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:51.046 7f8cf582b2c8 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table' fixed with properties {'gc_grace_seconds': 864000} 07/07/2025 00:44:51.051 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/07/2025 00:44:53.111 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/07/2025 00:44:53.112 7f8cf582b2c8 [contrail-device-manager] [WARNING]: table 'obj_shared_table', created 07/07/2025 00:44:53.117 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/07/2025 00:44:54.233 7f8cf2518048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:44:54.234 7f8cf2518048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:44:54.235 7f8cf2518048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:44:54.235 7f8cf2518048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:44:54.235 7f8cf2518048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 07/07/2025 00:44:54.235 7f8cf2518048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:44:54.235 7f8cf2518048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:44:54.236 7f8cf2518048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:44:54.236 7f8cf2518048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:54.236 7f8cf2518048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 07/07/2025 00:44:54.236 7f8cf2518048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:54.236 7f8cf2518048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:54.236 7f8cf2518048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.27:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:44:54.236 7f8cf2518048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:55.417 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/07/2025 00:44:56.417 7f8cf582b2c8 [contrail-device-manager] [WARNING]: table 'obj_shared_table' fixed with properties {'gc_grace_seconds': 864000} 07/07/2025 00:44:56.422 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/07/2025 00:44:56.422 7f8cedae9cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:56.422 7f8cedae9cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:44:56.422 7f8cedae9cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:56.458 7f8cf582b2c8 [contrail-device-manager] [ERROR]: Error in db walk read 'type' 07/07/2025 00:44:56.459 7f8cec3a0cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:56.459 7f8cec3a0cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Zookeeper:Zookeeper[] connection down >>, ] >> 07/07/2025 00:44:56.459 7f8cec3a0cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:56.467 7f8cec3a0d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:56.467 7f8cec3a0d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:44:56.467 7f8cec3a0d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:56.468 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 07/07/2025 00:44:56.468 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 07/07/2025 00:44:56.468 7f8cf58a1f60 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 07/07/2025 00:44:56.468 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3601-3. 07/07/2025 00:44:56.472 7f8cf58a1f60 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 07/07/2025 00:44:56.486 7f8cf58a1f60 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3601-3.'] 07/07/2025 00:44:56.495 7f8cf58a1f60 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 07/07/2025 00:44:56.500 7f8cf58a1f60 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 07/07/2025 00:44:56.510 7f8cf58a1f60 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3601-3.'] 07/07/2025 00:44:56.978 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Fetching all containers 07/07/2025 00:44:57.024 7f8cf582b2c8 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 07/07/2025 00:44:57.207 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3601-3..config_queue 07/07/2025 00:44:57.207 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3601-3..tftp_queue 07/07/2025 00:44:57.586 7f8cf58a1f60 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 07/07/2025 00:44:57.591 7f8cf58a1f60 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 07/07/2025 00:44:57.662 7f8cf58a1f60 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3601-3.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3601-3..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3601-3..tftp_queue'] 07/07/2025 00:44:57.708 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Process 140 prepared to run in Full mode... 07/07/2025 00:44:57.708 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 07/07/2025 00:44:57.708 7f8cf582b2c8 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 07/07/2025 00:44:57.711 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page 07/07/2025 00:44:57.711 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 07/07/2025 00:44:57.712 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page 07/07/2025 00:44:57.712 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 07/07/2025 00:44:57.712 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 07/07/2025 00:44:57.713 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 07/07/2025 00:44:57.716 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page 07/07/2025 00:44:57.716 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 07/07/2025 00:44:57.716 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 07/07/2025 00:44:57.716 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 07/07/2025 00:44:57.716 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 07/07/2025 00:44:57.716 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 07/07/2025 00:44:57.716 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 07/07/2025 00:44:57.716 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 07/07/2025 00:44:57.716 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 07/07/2025 00:44:57.717 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 07/07/2025 00:44:57.717 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 07/07/2025 00:44:57.717 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 07/07/2025 00:44:57.717 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 07/07/2025 00:44:57.718 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 07/07/2025 00:44:57.718 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 07/07/2025 00:44:57.718 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 07/07/2025 00:44:57.718 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 07/07/2025 00:44:57.719 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 07/07/2025 00:44:57.719 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 07/07/2025 00:44:57.719 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 07/07/2025 00:44:57.719 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page 07/07/2025 00:44:57.719 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 07/07/2025 00:44:57.719 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 07/07/2025 00:44:57.719 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 07/07/2025 00:44:57.720 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 07/07/2025 00:44:57.720 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 07/07/2025 00:44:57.720 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page 07/07/2025 00:44:57.720 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 07/07/2025 00:44:57.720 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 07/07/2025 00:44:57.721 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 07/07/2025 00:44:57.721 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 07/07/2025 00:44:57.721 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 07/07/2025 00:44:57.721 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 07/07/2025 00:44:57.721 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 07/07/2025 00:44:57.721 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 07/07/2025 00:44:57.721 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 07/07/2025 00:44:57.721 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 07/07/2025 00:44:57.721 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 07/07/2025 00:44:57.721 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 07/07/2025 00:44:57.721 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 07/07/2025 00:44:57.721 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 07/07/2025 00:44:57.724 7f8cf582b2c8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 07/07/2025 00:44:57.735 7f8cec406438 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 07/07/2025 00:44:57.736 7f8cec406438 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 07/07/2025 00:44:57.784 7f8ce7b1a438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:57.784 7f8ce7b1a438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 07/07/2025 00:44:57.784 7f8ce7b1a438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:57.795 7f8ce79ecd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:57.795 7f8ce79ecd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 07/07/2025 00:44:57.796 7f8ce79ecd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:58.236 7f8cecd5b2e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:44:58.237 7f8cecd5b2e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:44:58.237 7f8cecd5b2e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:44:58.237 7f8cecd5b2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:44:58.237 7f8cecd5b2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 07/07/2025 00:44:58.237 7f8cecd5b2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:44:58.237 7f8cecd5b2e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:44:58.237 7f8cecd5b2e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:44:58.237 7f8cecd5b2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:58.238 7f8cecd5b2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 07/07/2025 00:44:58.238 7f8cecd5b2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:58.238 7f8cecd5b2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:58.238 7f8cecd5b2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:44:58.238 7f8cecd5b2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:44:58.684 7f8cf58a1f60 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 07/07/2025 00:44:58.688 7f8cf58a1f60 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 07/07/2025 00:44:58.756 7f8cf58a1f60 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3601-3.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3601-3..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3601-3..tftp_queue', 'device_manager_ztp.ztp_queue'] 07/07/2025 00:45:02.238 7f8ce799c2e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:45:02.239 7f8ce799c2e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:45:02.239 7f8ce799c2e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:45:02.239 7f8ce799c2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:02.239 7f8ce799c2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 07/07/2025 00:45:02.240 7f8ce799c2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:02.240 7f8ce799c2e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:45:02.240 7f8ce799c2e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:45:02.240 7f8ce799c2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:02.240 7f8ce799c2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 07/07/2025 00:45:02.240 7f8ce799c2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:02.240 7f8ce799c2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:02.240 7f8ce799c2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:45:02.240 7f8ce799c2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:03.911 7f8ce79df8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:03.911 7f8ce79df8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:45:03.911 7f8ce79df8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:06.240 7f8ce7911828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:45:06.241 7f8ce7911828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:45:06.241 7f8ce7911828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:45:06.241 7f8ce7911828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:06.242 7f8ce7911828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/07/2025 00:45:06.242 7f8ce7911828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:06.242 7f8ce7911828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:45:06.242 7f8ce7911828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:45:06.242 7f8ce7911828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:06.242 7f8ce7911828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:45:06.242 7f8ce7911828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:06.242 7f8ce7911828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:06.242 7f8ce7911828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.27:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:45:06.242 7f8ce7911828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:10.242 7f8ce78c1e10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:45:10.243 7f8ce78c1e10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:45:10.243 7f8ce78c1e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:45:10.243 7f8ce78c1e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:10.243 7f8ce78c1e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/07/2025 00:45:10.243 7f8ce78c1e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:10.243 7f8ce78c1e10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:45:10.244 7f8ce78c1e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:45:10.244 7f8ce78c1e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:10.244 7f8ce78c1e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:45:10.244 7f8ce78c1e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:10.244 7f8ce78c1e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:10.244 7f8ce78c1e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:45:10.244 7f8ce78c1e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:14.244 7f8ce794bcc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:45:14.245 7f8ce794bcc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:45:14.245 7f8ce794bcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:45:14.245 7f8ce794bcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:14.246 7f8ce794bcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/07/2025 00:45:14.246 7f8ce794bcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:14.246 7f8ce794bcc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:45:14.246 7f8ce794bcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:45:14.246 7f8ce794bcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:14.246 7f8ce794bcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:45:14.246 7f8ce794bcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:14.246 7f8ce794bcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:14.246 7f8ce794bcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:45:14.246 7f8ce794bcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:18.246 7f8ce794bcc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:45:18.250 7f8ce794bcc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:45:18.250 7f8ce794bcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:45:18.250 7f8ce794bcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:18.251 7f8ce794bcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/07/2025 00:45:18.251 7f8ce794bcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:18.251 7f8ce794bcc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:45:18.251 7f8ce794bcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:45:18.251 7f8ce794bcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:18.251 7f8ce794bcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:45:18.251 7f8ce794bcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:18.251 7f8ce794bcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:18.251 7f8ce794bcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.27:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:45:18.251 7f8ce794bcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:22.251 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:45:22.252 7f8cec4154e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:45:22.253 7f8cec4154e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:45:22.253 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:22.253 7f8cec4154e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/07/2025 00:45:22.253 7f8cec4154e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:22.253 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:45:22.253 7f8cec4154e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:45:22.254 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:22.254 7f8cec4154e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:45:22.254 7f8cec4154e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:22.254 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:22.254 7f8cec4154e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:45:22.254 7f8cec4154e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:26.254 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:45:26.255 7f8cec4154e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:45:26.255 7f8cec4154e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:45:26.255 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:26.255 7f8cec4154e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/07/2025 00:45:26.255 7f8cec4154e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:26.255 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:45:26.256 7f8cec4154e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:45:26.256 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:26.256 7f8cec4154e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:45:26.256 7f8cec4154e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:26.256 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:26.256 7f8cec4154e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:45:26.256 7f8cec4154e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:30.256 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:45:30.257 7f8cec4154e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:45:30.257 7f8cec4154e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:45:30.257 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:30.257 7f8cec4154e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/07/2025 00:45:30.257 7f8cec4154e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:30.257 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:45:30.257 7f8cec4154e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:45:30.258 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:30.258 7f8cec4154e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:45:30.258 7f8cec4154e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:30.258 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:30.258 7f8cec4154e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.27:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:45:30.258 7f8cec4154e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:34.258 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:45:34.259 7f8cec4154e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:45:34.259 7f8cec4154e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:45:34.259 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:34.259 7f8cec4154e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/07/2025 00:45:34.259 7f8cec4154e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:34.259 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:45:34.259 7f8cec4154e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:45:34.259 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:34.260 7f8cec4154e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:45:34.260 7f8cec4154e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:34.260 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:34.260 7f8cec4154e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:45:34.260 7f8cec4154e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:38.259 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:45:38.261 7f8cec4154e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:45:38.261 7f8cec4154e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:45:38.261 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:38.261 7f8cec4154e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/07/2025 00:45:38.261 7f8cec4154e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:38.261 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:45:38.262 7f8cec4154e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:45:38.262 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:38.262 7f8cec4154e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:45:38.262 7f8cec4154e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:38.262 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:38.262 7f8cec4154e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:45:38.262 7f8cec4154e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:42.262 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:45:42.263 7f8cec4154e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:45:42.264 7f8cec4154e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:45:42.264 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:42.264 7f8cec4154e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/07/2025 00:45:42.264 7f8cec4154e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:42.264 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:45:42.264 7f8cec4154e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:45:42.264 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:42.265 7f8cec4154e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:45:42.265 7f8cec4154e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:42.265 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:42.265 7f8cec4154e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.27:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:45:42.265 7f8cec4154e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:46.264 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:45:46.265 7f8cec4154e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:45:46.265 7f8cec4154e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:45:46.265 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:46.265 7f8cec4154e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/07/2025 00:45:46.265 7f8cec4154e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:46.265 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:45:46.265 7f8cec4154e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:45:46.265 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:46.266 7f8cec4154e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:45:46.266 7f8cec4154e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:46.266 7f8cec4154e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:46.266 7f8cec4154e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:45:46.266 7f8cec4154e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:50.265 7f8ce78c1e10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:45:50.266 7f8ce78c1e10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:45:50.267 7f8ce78c1e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:45:50.267 7f8ce78c1e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:50.267 7f8ce78c1e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/07/2025 00:45:50.267 7f8ce78c1e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:50.267 7f8ce78c1e10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:45:50.267 7f8ce78c1e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:45:50.267 7f8ce78c1e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:50.268 7f8ce78c1e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:45:50.268 7f8ce78c1e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:50.268 7f8ce78c1e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:50.268 7f8ce78c1e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:45:50.268 7f8ce78c1e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:54.267 7f8ce78c18d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:45:54.268 7f8ce78c18d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:45:54.269 7f8ce78c18d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:45:54.269 7f8ce78c18d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:54.269 7f8ce78c18d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/07/2025 00:45:54.269 7f8ce78c18d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:54.269 7f8ce78c18d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:45:54.269 7f8ce78c18d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:45:54.269 7f8ce78c18d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:54.269 7f8ce78c18d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:45:54.269 7f8ce78c18d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:54.270 7f8ce78c18d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:54.270 7f8ce78c18d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.27:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:45:54.270 7f8ce78c18d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:58.269 7f8ce7910ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:45:58.270 7f8ce7910ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:45:58.271 7f8ce7910ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:45:58.271 7f8ce7910ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:58.271 7f8ce7910ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/07/2025 00:45:58.271 7f8ce7910ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:45:58.271 7f8ce7910ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:45:58.271 7f8ce7910ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:45:58.271 7f8ce7910ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:58.271 7f8ce7910ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:45:58.272 7f8ce7910ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:58.272 7f8ce7910ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:45:58.272 7f8ce7910ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:45:58.272 7f8ce7910ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:02.271 7f8cf297ac18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:46:02.272 7f8cf297ac18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:46:02.272 7f8cf297ac18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:46:02.272 7f8cf297ac18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:02.273 7f8cf297ac18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/07/2025 00:46:02.273 7f8cf297ac18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:02.273 7f8cf297ac18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:46:02.273 7f8cf297ac18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:46:02.274 7f8cf297ac18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:02.274 7f8cf297ac18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:46:02.274 7f8cf297ac18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:02.274 7f8cf297ac18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:02.274 7f8cf297ac18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:46:02.274 7f8cf297ac18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:06.273 7f8cf2987630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:46:06.274 7f8cf2987630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:46:06.274 7f8cf2987630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:46:06.274 7f8cf2987630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:06.274 7f8cf2987630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/07/2025 00:46:06.274 7f8cf2987630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:06.274 7f8cf2987630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:46:06.275 7f8cf2987630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:46:06.275 7f8cf2987630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:06.275 7f8cf2987630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:46:06.275 7f8cf2987630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:06.275 7f8cf2987630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:06.275 7f8cf2987630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.27:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:46:06.275 7f8cf2987630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:10.275 7f8cec425978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:46:10.278 7f8cec425978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:46:10.278 7f8cec425978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:46:10.278 7f8cec425978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:10.278 7f8cec425978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/07/2025 00:46:10.278 7f8cec425978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:10.278 7f8cec425978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:46:10.279 7f8cec425978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:46:10.279 7f8cec425978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:10.279 7f8cec425978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:46:10.279 7f8cec425978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:10.279 7f8cec425978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:10.279 7f8cec425978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:46:10.279 7f8cec425978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:14.279 7f8cec406828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:46:14.280 7f8cec406828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:46:14.281 7f8cec406828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:46:14.281 7f8cec406828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:14.281 7f8cec406828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/07/2025 00:46:14.281 7f8cec406828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:14.281 7f8cec406828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:46:14.281 7f8cec406828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:46:14.281 7f8cec406828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:14.281 7f8cec406828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:46:14.281 7f8cec406828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:14.281 7f8cec406828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:14.281 7f8cec406828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:46:14.282 7f8cec406828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:18.281 7f8ce78a8198 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:46:18.283 7f8ce78a8198 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:46:18.283 7f8ce78a8198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:46:18.283 7f8ce78a8198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:18.283 7f8ce78a8198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/07/2025 00:46:18.283 7f8ce78a8198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:18.283 7f8ce78a8198 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:46:18.283 7f8ce78a8198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:46:18.284 7f8ce78a8198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:18.284 7f8ce78a8198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:46:18.284 7f8ce78a8198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:18.284 7f8ce78a8198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:18.284 7f8ce78a8198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.27:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:46:18.284 7f8ce78a8198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:21.442 7f8cf582b2c8 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 07/07/2025 00:46:21.443 7f8cf582b2c8 [contrail-device-manager] [WARNING]: initializing cassandra cluster 07/07/2025 00:46:21.443 7f8cf582b2c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 07/07/2025 00:46:21.443 7f8cf3bc7240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:21.444 7f8cf3bc7240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 07/07/2025 00:46:21.444 7f8cf3bc7240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:21.445 7f8cf582b2c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 07/07/2025 00:46:21.491 7f8cf582b2c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 07/07/2025 00:46:21.493 7f8ce7a9e240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:21.494 7f8ce7a9e240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Initializing description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 07/07/2025 00:46:21.494 7f8ce7a9e240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:21.502 7f8cf582b2c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 07/07/2025 00:46:21.502 7f8ce7a9e048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:21.502 7f8ce7a9e048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 07/07/2025 00:46:21.503 7f8ce7a9e048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:21.514 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/07/2025 00:46:22.283 7f8cf297ac18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:46:22.284 7f8cf297ac18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:46:22.285 7f8cf297ac18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:46:22.285 7f8cf297ac18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:22.285 7f8cf297ac18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 07/07/2025 00:46:22.285 7f8cf297ac18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:22.285 7f8cf297ac18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:46:22.288 7f8cf297ac18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:46:22.304 7f8cf297ac18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:22.304 7f8cf297ac18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 07/07/2025 00:46:22.305 7f8cf297ac18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:22.305 7f8cf297ac18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:22.305 7f8cf297ac18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:46:22.305 7f8cf297ac18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:22.384 7f8cf582b2c8 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace', created 07/07/2025 00:46:22.398 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/07/2025 00:46:22.406 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/07/2025 00:46:23.264 7f8cf582b2c8 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None} 07/07/2025 00:46:23.268 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/07/2025 00:46:23.268 7f8cf582b2c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 07/07/2025 00:46:23.338 7f8cf582b2c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 07/07/2025 00:46:23.344 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/07/2025 00:46:24.300 7f8cf582b2c8 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table', created 07/07/2025 00:46:24.305 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/07/2025 00:46:24.310 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/07/2025 00:46:25.440 7f8cf582b2c8 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table' fixed with properties {'gc_grace_seconds': 864000} 07/07/2025 00:46:25.443 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/07/2025 00:46:25.448 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/07/2025 00:46:26.284 7f8ce7893240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:46:26.285 7f8ce7893240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:46:26.286 7f8ce7893240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:46:26.286 7f8ce7893240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:26.286 7f8ce7893240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 07/07/2025 00:46:26.286 7f8ce7893240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:26.286 7f8ce7893240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:46:26.286 7f8ce7893240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:46:26.286 7f8ce7893240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:26.286 7f8ce7893240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 07/07/2025 00:46:26.287 7f8ce7893240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:26.287 7f8ce7893240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:26.287 7f8ce7893240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:46:26.287 7f8ce7893240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:26.380 7f8cf582b2c8 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table', created 07/07/2025 00:46:26.384 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/07/2025 00:46:26.389 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/07/2025 00:46:27.281 7f8cf582b2c8 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table' fixed with properties {'gc_grace_seconds': 864000} 07/07/2025 00:46:27.285 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/07/2025 00:46:27.292 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/07/2025 00:46:28.330 7f8cf582b2c8 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table', created 07/07/2025 00:46:28.336 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/07/2025 00:46:28.348 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/07/2025 00:46:29.432 7f8cf582b2c8 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table' fixed with properties {'gc_grace_seconds': 864000} 07/07/2025 00:46:29.437 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/07/2025 00:46:29.444 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/07/2025 00:46:30.165 7f8cf582b2c8 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table', created 07/07/2025 00:46:30.169 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/07/2025 00:46:30.175 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/07/2025 00:46:30.286 7f8cedad34e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:46:30.287 7f8cedad34e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:46:30.287 7f8cedad34e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:46:30.287 7f8cedad34e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:30.287 7f8cedad34e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 07/07/2025 00:46:30.287 7f8cedad34e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:30.287 7f8cedad34e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:46:30.288 7f8cedad34e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:46:30.288 7f8cedad34e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:30.288 7f8cedad34e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 07/07/2025 00:46:30.288 7f8cedad34e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:30.288 7f8cedad34e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:30.288 7f8cedad34e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.27:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:46:30.288 7f8cedad34e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:31.324 7f8cf582b2c8 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table' fixed with properties {'gc_grace_seconds': 864000} 07/07/2025 00:46:31.331 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/07/2025 00:46:31.331 7f8ce795d780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:31.332 7f8ce795d780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:46:31.332 7f8ce795d780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:31.375 7f8cf582b2c8 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.41:5673//', 'pyamqp://guest:guest@10.0.0.27:5673//', 'pyamqp://guest:guest@10.0.0.23:5673//'] 07/07/2025 00:46:31.376 7f8cf582b2c8 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 07/07/2025 00:46:31.376 7f8ce786dc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:31.376 7f8ce786dc18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Initializing description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 07/07/2025 00:46:31.376 7f8ce786dc18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:31.376 7f8ce786dc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:31.376 7f8ce786dc18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 07/07/2025 00:46:31.376 7f8ce786dc18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:31.380 7f8cf582b2c8 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 07/07/2025 00:46:31.381 7f8ce786dc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:31.381 7f8ce786dc18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:46:31.381 7f8ce786dc18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:31.475 7f8cf582b2c8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 07/07/2025 00:46:31.476 7f8ce786dd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:31.476 7f8ce786dd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 07/07/2025 00:46:31.476 7f8ce786dd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:31.484 7f8cf582b2c8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 07/07/2025 00:46:31.485 7f8ce7882e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:31.485 7f8ce7882e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:46:31.485 7f8ce7882e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:31.486 7f8ce789d6d8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 07/07/2025 00:46:31.486 7f8ce799f6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:31.486 7f8ce799f6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 07/07/2025 00:46:31.486 7f8ce799f6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:31.496 7f8ce789d6d8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 07/07/2025 00:46:31.496 7f8ce786d048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:31.496 7f8ce786d048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:46:31.496 7f8ce786d048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:34.288 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:46:34.289 7f8ce795de10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:46:34.289 7f8ce795de10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:46:34.289 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:34.289 7f8ce795de10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/07/2025 00:46:34.289 7f8ce795de10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:34.289 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:46:34.289 7f8ce795de10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:46:34.290 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:34.290 7f8ce795de10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:46:34.290 7f8ce795de10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:34.290 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:34.290 7f8ce795de10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:46:34.290 7f8ce795de10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:38.290 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:46:38.291 7f8ce795de10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:46:38.291 7f8ce795de10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:46:38.291 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:38.292 7f8ce795de10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/07/2025 00:46:38.292 7f8ce795de10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:38.292 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:46:38.292 7f8ce795de10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:46:38.292 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:38.292 7f8ce795de10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:46:38.292 7f8ce795de10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:38.292 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:38.292 7f8ce795de10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:46:38.293 7f8ce795de10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:42.293 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:46:42.294 7f8ce795de10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:46:42.294 7f8ce795de10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:46:42.294 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:42.294 7f8ce795de10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/07/2025 00:46:42.294 7f8ce795de10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:42.294 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:46:42.295 7f8ce795de10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:46:42.295 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:42.295 7f8ce795de10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:46:42.295 7f8ce795de10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:42.295 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:42.295 7f8ce795de10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.27:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:46:42.295 7f8ce795de10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:46.295 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:46:46.297 7f8ce795de10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:46:46.297 7f8ce795de10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:46:46.297 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:46.297 7f8ce795de10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/07/2025 00:46:46.297 7f8ce795de10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:46.297 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:46:46.298 7f8ce795de10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:46:46.298 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:46.298 7f8ce795de10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:46:46.298 7f8ce795de10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:46.298 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:46.298 7f8ce795de10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:46:46.298 7f8ce795de10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:50.297 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:46:50.298 7f8ce795de10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/07/2025 00:46:50.299 7f8ce795de10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:46:50.299 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:50.299 7f8ce795de10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/07/2025 00:46:50.299 7f8ce795de10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:50.299 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/07/2025 00:46:50.299 7f8ce795de10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/07/2025 00:46:50.299 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:50.300 7f8ce795de10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/07/2025 00:46:50.300 7f8ce795de10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:50.300 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:50.300 7f8ce795de10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:46:50.300 7f8ce795de10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:46:54.300 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:46:54.301 7f8ce795de10 [contrail-device-manager] [INFO]: Session Event: TCP Connected 07/07/2025 00:46:54.301 7f8ce795de10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:46:54.301 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:54.301 7f8ce795de10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/07/2025 00:46:54.301 7f8ce795de10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:46:54.301 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect] 07/07/2025 00:46:54.301 7f8ce795de10 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19 07/07/2025 00:46:54.302 7f8ce795de10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 07/07/2025 00:46:54.302 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 07/07/2025 00:46:54.302 7f8ce795de10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 07/07/2025 00:46:54.302 7f8ce77a7198 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 07/07/2025 00:46:54.303 7f8ce77a7198 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 07/07/2025 00:46:54.485 7f8ce795d0f0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3601-2.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.27', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3601-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3601-2.'], 'id_perms': {'created': '2025-07-07T00:46:54.470341', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-07-07T00:46:54.470341', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13772006735942750945, 'uuid_mslong': 4543757591188491181}}, 'parent_type': 'global-system-config', 'parent_uuid': '39ea06e2-7b78-4b3f-91a9-7192d68ee64b', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '3f0eab74-9046-4bad-bf1f-ff28bc2792e1'}, 'oper': 'CREATE', 'request-id': 'req-2785f589-4b1c-42a8-a176-86477615c515', 'type': 'analytics_node', 'uuid': '3f0eab74-9046-4bad-bf1f-ff28bc2792e1'} 07/07/2025 00:46:54.953 7f8ce795d0f0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3601-3.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.23', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3601-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3601-3.'], 'id_perms': {'created': '2025-07-07T00:46:54.947152', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-07-07T00:46:54.947152', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13311781150781077648, 'uuid_mslong': 553546533502995809}}, 'parent_type': 'global-system-config', 'parent_uuid': '39ea06e2-7b78-4b3f-91a9-7192d68ee64b', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '07ae97a3-b018-4d61-b8bc-f26d840bf890'}, 'oper': 'CREATE', 'request-id': 'req-a49bf303-484e-49bf-9604-06101c90dcf9', 'type': 'analytics_node', 'uuid': '07ae97a3-b018-4d61-b8bc-f26d840bf890'} 07/07/2025 00:46:55.045 7f8ce795d0f0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3601-1.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.41', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3601-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3601-1.'], 'id_perms': {'created': '2025-07-07T00:46:55.034534', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-07-07T00:46:55.034534', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9542702172950930212, 'uuid_mslong': 17249535080905657138}}, 'parent_type': 'global-system-config', 'parent_uuid': '39ea06e2-7b78-4b3f-91a9-7192d68ee64b', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'ef62a8c3-9e05-4b32-846e-7d112016a324'}, 'oper': 'CREATE', 'request-id': 'req-4de8e448-ebb4-45b4-954b-9d23581df137', 'type': 'analytics_node', 'uuid': 'ef62a8c3-9e05-4b32-846e-7d112016a324'} 07/07/2025 00:47:05.590 7f8ce77a7eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connection Closed 07/07/2025 00:47:05.591 7f8cec42c630 [contrail-device-manager] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 07/07/2025 00:47:05.592 7f8cec42c630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 07/07/2025 00:47:05.592 7f8cec42c630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:47:05.592 7f8cec42c630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.27:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 07/07/2025 00:47:05.592 7f8cec42c630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:47:05.592 7f8cec42c630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:47:05.592 7f8cec42c630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 1 pid = 140 http_port = 8096 start_time = 1751849078226926 collector_name = collector_ip = 10.0.0.27:8086 collector_list = [ 10.0.0.27:8086, 10.0.0.23:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/07/2025 00:47:05.592 7f8cec42c630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/07/2025 00:47:09.593 7f8cec42c630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/07/2025 00:47:09.594 7f8cec42c630 [contrail-device-manager] [INFO]: Session Event: TCP Connected 07/07/2025 00:47:09.594 7f8cec42c630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/07/2025 00:47:09.594 7f8cec42c630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:47:09.594 7f8cec42c630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3601-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.27:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.27:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.27:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.27,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/07/2025 00:47:09.594 7f8cec42c630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/07/2025 00:47:09.594 7f8cec42c630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect] 07/07/2025 00:47:09.594 7f8cec42c630 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19 07/07/2025 00:47:09.595 7f8cec42c630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 07/07/2025 00:47:09.595 7f8cec42c630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 07/07/2025 00:47:09.595 7f8cec42c630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 07/07/2025 00:47:09.595 7f8cec3a36d8 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 07/07/2025 00:47:09.596 7f8cec3a36d8 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 07/07/2025 00:47:09.609 7f8ce799c2e8 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001017 07/07/2025 00:47:09.609 7f8ce799c2e8 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 07/07/2025 00:47:09.609 7f8cec42c630 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 07/07/2025 00:47:09.609 7f8cec42c630 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 0 07/07/2025 00:47:09.610 7f8cec42c630 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 07/07/2025 00:47:09.610 7f8cec42c630 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 07/07/2025 00:47:09.610 7f8cec42c630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 07/07/2025 00:47:09.610 7f8cf3294b70 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 07/07/2025 00:47:09.611 7f8cf3294b70 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 07/07/2025 00:47:11.831 7f8ce795d0f0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3601-3.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.23', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3601-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3601-3.'], 'id_perms': {'created': '2025-07-07T00:47:11.797788', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-07-07T00:47:11.797788', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11968508496307764804, 'uuid_mslong': 7528006245854236075}}, 'parent_type': 'global-system-config', 'parent_uuid': '39ea06e2-7b78-4b3f-91a9-7192d68ee64b', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '6878d9cb-a710-4dab-a618-af10c5ab3e44'}, 'oper': 'CREATE', 'request-id': 'req-feb613e2-4974-4780-820e-e6eb625f62a2', 'type': 'analytics_alarm_node', 'uuid': '6878d9cb-a710-4dab-a618-af10c5ab3e44'} 07/07/2025 00:47:11.925 7f8ce795d0f0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3601-1.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.41', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3601-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3601-1.'], 'id_perms': {'created': '2025-07-07T00:47:11.901866', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-07-07T00:47:11.901866', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11137768275668432135, 'uuid_mslong': 3335051162484426746}}, 'parent_type': 'global-system-config', 'parent_uuid': '39ea06e2-7b78-4b3f-91a9-7192d68ee64b', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '2e487b5c-c7a3-47fa-9a91-4d3c7dd4f507'}, 'oper': 'CREATE', 'request-id': 'req-0a2177b8-5ef4-4f2a-a0c1-7436e40fd432', 'type': 'analytics_alarm_node', 'uuid': '2e487b5c-c7a3-47fa-9a91-4d3c7dd4f507'} 07/07/2025 00:47:11.959 7f8ce795d0f0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3601-2.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.27', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3601-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3601-2.'], 'id_perms': {'created': '2025-07-07T00:47:11.945549', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-07-07T00:47:11.945549', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9798948152377523618, 'uuid_mslong': 14645982623816436306}}, 'parent_type': 'global-system-config', 'parent_uuid': '39ea06e2-7b78-4b3f-91a9-7192d68ee64b', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'cb40fb99-3f1f-4a52-87fc-db6d72eebda2'}, 'oper': 'CREATE', 'request-id': 'req-f90bba53-e206-4c9e-b568-e661bcaed5b5', 'type': 'analytics_alarm_node', 'uuid': 'cb40fb99-3f1f-4a52-87fc-db6d72eebda2'} 07/07/2025 00:47:16.530 7f8ce799c2e8 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001296 07/07/2025 00:47:16.531 7f8ce799c2e8 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 07/07/2025 00:47:16.531 7f8ce7909ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 07/07/2025 00:47:16.531 7f8ce7909ac8 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 2 07/07/2025 00:47:16.531 7f8ce7909ac8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 07/07/2025 00:47:16.531 7f8ce7909ac8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 07/07/2025 00:47:22.015 7f8ce795d0f0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3601-2.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.27', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3601-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3601-2.'], 'id_perms': {'created': '2025-07-07T00:47:21.999041', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-07-07T00:47:21.999041', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10115644678671673317, 'uuid_mslong': 7333402042719555141}}, 'parent_type': 'global-system-config', 'parent_uuid': '39ea06e2-7b78-4b3f-91a9-7192d68ee64b', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '65c57a4d-c76c-4e45-8c61-fd3d824ba7e5'}, 'oper': 'CREATE', 'request-id': 'req-332e0124-9f5e-4b63-8bdb-78173533242a', 'type': 'analytics_snmp_node', 'uuid': '65c57a4d-c76c-4e45-8c61-fd3d824ba7e5'} 07/07/2025 00:47:22.056 7f8ce795d0f0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3601-3.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.23', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3601-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3601-3.'], 'id_perms': {'created': '2025-07-07T00:47:22.035255', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-07-07T00:47:22.035255', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10144371652435205906, 'uuid_mslong': 3545353801856272009}}, 'parent_type': 'global-system-config', 'parent_uuid': '39ea06e2-7b78-4b3f-91a9-7192d68ee64b', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '3133a07f-aab9-4689-8cc8-0c454cf15f12'}, 'oper': 'CREATE', 'request-id': 'req-b625e6a7-39da-4642-94f4-74060093e210', 'type': 'analytics_snmp_node', 'uuid': '3133a07f-aab9-4689-8cc8-0c454cf15f12'} 07/07/2025 00:47:22.135 7f8ce795d0f0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3601-1.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.41', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3601-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3601-1.'], 'id_perms': {'created': '2025-07-07T00:47:22.113919', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-07-07T00:47:22.113919', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13702184582456632564, 'uuid_mslong': 2258303181989298333}}, 'parent_type': 'global-system-config', 'parent_uuid': '39ea06e2-7b78-4b3f-91a9-7192d68ee64b', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '1f571ac7-6d9b-409d-be27-f047cc2964f4'}, 'oper': 'CREATE', 'request-id': 'req-ec9fd581-bfcb-4615-b6ba-7c36c66bbc95', 'type': 'analytics_snmp_node', 'uuid': '1f571ac7-6d9b-409d-be27-f047cc2964f4'} 07/07/2025 00:49:02.312 7f8ce795d0f0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-9406e957-49a9-4154-ba84-2fb4412de023', 'type': 'global_system_config', 'uuid': '39ea06e2-7b78-4b3f-91a9-7192d68ee64b'} 07/07/2025 00:49:02.352 7f8ce795d0f0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-0ee5845f-b4e5-4031-8c28-a2e79a6c1e5f', 'type': 'global_system_config', 'uuid': '39ea06e2-7b78-4b3f-91a9-7192d68ee64b'} 07/07/2025 00:49:02.455 7f8ce795d0f0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3601-1'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-3601-1', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3601-1'], 'id_perms': {'created': '2025-07-07T00:49:02.443939', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-07-07T00:49:02.443939', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11991472854060338322, 'uuid_mslong': 18304612510686659251}}, 'parent_type': 'global-system-config', 'parent_uuid': '39ea06e2-7b78-4b3f-91a9-7192d68ee64b', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'fe070c18-b2a6-42b3-a66a-4506f8018c92', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.193'}, 'oper': 'CREATE', 'request-id': 'req-1fc3ea12-eb71-4124-873d-7a0f1e39c1ff', 'type': 'virtual_router', 'uuid': 'fe070c18-b2a6-42b3-a66a-4506f8018c92'} 07/07/2025 00:49:02.486 7f8ce795d0f0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3601-2'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-3601-2', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3601-2'], 'id_perms': {'created': '2025-07-07T00:49:02.475794', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-07-07T00:49:02.475794', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12716123447075312900, 'uuid_mslong': 16262421649519561192}}, 'parent_type': 'global-system-config', 'parent_uuid': '39ea06e2-7b78-4b3f-91a9-7192d68ee64b', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'e1afba54-073c-49e8-b078-bee6f6f09504', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.129'}, 'oper': 'CREATE', 'request-id': 'req-d114c961-becb-4192-bf13-f492403c573e', 'type': 'virtual_router', 'uuid': 'e1afba54-073c-49e8-b078-bee6f6f09504'} 07/07/2025 00:49:02.821 7f8ce795d0f0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3601-1', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3601-1', 'vhost0'], 'id_perms': {'created': '2025-07-07T00:49:02.794923', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-07-07T00:49:02.794923', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10843679042805270387, 'uuid_mslong': 14882158667782963201}}, 'parent_type': 'virtual-router', 'parent_uuid': 'fe070c18-b2a6-42b3-a66a-4506f8018c92', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'ce880c75-9434-4001-967c-7ca31cc18773', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:ce:88:0c:75:94']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': 'd652dc3a-abe1-4bd7-ae1b-24241ad0a951'}]}, 'oper': 'CREATE', 'request-id': 'req-33cbae77-54cc-45b6-8a5b-dd38e9da3a7e', 'type': 'virtual_machine_interface', 'uuid': 'ce880c75-9434-4001-967c-7ca31cc18773'} 07/07/2025 00:49:02.881 7f8ce795d0f0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3601-2', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3601-2', 'vhost0'], 'id_perms': {'created': '2025-07-07T00:49:02.860240', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-07-07T00:49:02.860240', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12762929167301197011, 'uuid_mslong': 1277156566222851830}}, 'parent_type': 'virtual-router', 'parent_uuid': 'e1afba54-073c-49e8-b078-bee6f6f09504', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '11b95f21-5c45-4ef6-b11f-0874ee75bcd3', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:11:b9:5f:21:5c']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': 'd652dc3a-abe1-4bd7-ae1b-24241ad0a951'}]}, 'oper': 'CREATE', 'request-id': 'req-08f511ca-e353-43b7-94a2-4f07bbfabb9f', 'type': 'virtual_machine_interface', 'uuid': '11b95f21-5c45-4ef6-b11f-0874ee75bcd3'} 07/07/2025 00:49:02.894 7f8ce795d0f0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3601-1', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-33cbae77-54cc-45b6-8a5b-dd38e9da3a7e', 'type': 'virtual_machine_interface', 'uuid': 'ce880c75-9434-4001-967c-7ca31cc18773'} 07/07/2025 00:49:02.953 7f8ce795d0f0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3601-2', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-08f511ca-e353-43b7-94a2-4f07bbfabb9f', 'type': 'virtual_machine_interface', 'uuid': '11b95f21-5c45-4ef6-b11f-0874ee75bcd3'} 07/07/2025 00:49:03.359 7f8ce795d0f0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-85d80357-201a-4c2f-99c2-39530e186ba8', 'type': 'access_control_list', 'uuid': '12ffe9e4-2193-49d4-94b0-b8d488fa915d'} 07/07/2025 00:49:17.643 7f8ce786fe10 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 07/07/2025 00:49:54.557 7f8cf47db828 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 07/07/2025 01:42:08.130 7f8ce77a4588 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map