11/10/2025 05:02:22.145 7f2f9deda748 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 11/10/2025 05:02:22.145 7f2f9deda748 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 11/10/2025 05:02:22.145 7f2f9deda748 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 11/10/2025 05:02:22.145 7f2f9557f048 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 11/10/2025 05:02:22.146 7f2f9557f048 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 11/10/2025 05:02:22.146 7f2f9557f048 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/10/2025 05:02:22.146 7f2f9557f048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 11/10/2025 05:02:22.146 7f2f9557f048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:22.147 7f2f9557f048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/10/2025 05:02:22.147 7f2f9557f048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:22.147 7f2f9557f048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:22.147 7f2f9557f048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1762750942146295 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 11/10/2025 05:02:22.147 7f2f9557f048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:22.197 7f2f9deda748 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//', 'pyamqp://guest:guest@10.0.0.33:5673//'] 11/10/2025 05:02:22.209 7f2f9deda748 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 11/10/2025 05:02:22.209 7f2f954f52e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:22.210 7f2f954f52e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/10/2025 05:02:22.210 7f2f954f52e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:22.210 7f2f954f52e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:22.210 7f2f954f52e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/10/2025 05:02:22.210 7f2f954f52e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:22.215 7f2f9deda748 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 11/10/2025 05:02:22.216 7f2f954f52e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:22.216 7f2f954f52e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/10/2025 05:02:22.216 7f2f954f52e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:22.229 7f2f9deda748 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 11/10/2025 05:02:22.230 7f2f954fc390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:22.230 7f2f954fc390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/10/2025 05:02:22.230 7f2f954fc390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:22.235 7f2f9deda748 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 11/10/2025 05:02:22.235 7f2fa3d80978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:22.235 7f2fa3d80978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/10/2025 05:02:22.235 7f2fa3d80978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:22.247 7f2f9deda748 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 11/10/2025 05:02:22.247 7f2f9deda748 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 11/10/2025 05:02:22.248 7f2f9deda748 [contrail-device-manager] [WARNING]: initializing cassandra cluster 11/10/2025 05:02:22.248 7f2f9deda748 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 11/10/2025 05:02:22.248 7f2fa3cda630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 11/10/2025 05:02:22.249 7f2fa3cda630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/10/2025 05:02:22.249 7f2f954fc438 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 11/10/2025 05:02:22.249 7f2f954fc438 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 11/10/2025 05:02:22.249 7f2f9536f048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:22.249 7f2f9536f048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/10/2025 05:02:22.249 7f2f9536f048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:22.251 7f2f9deda748 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 11/10/2025 05:02:22.255 7f2f954fc438 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 11/10/2025 05:02:22.256 7f2fa3cda630 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/10/2025 05:02:22.256 7f2fa3cda630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 11/10/2025 05:02:22.300 7f2f9deda748 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 11/10/2025 05:02:22.303 7f2f9534a2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:22.304 7f2f9534a2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 11/10/2025 05:02:22.304 7f2f9534a2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:22.312 7f2f9deda748 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 11/10/2025 05:02:22.312 7f2f95346588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:22.312 7f2f95346588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/10/2025 05:02:22.312 7f2f95346588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:22.323 7f2f9deda748 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 11/10/2025 05:02:22.346 7f2f9deda748 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 11/10/2025 05:02:24.015 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/10/2025 05:02:24.879 7f2f9deda748 [contrail-device-manager] [WARNING]: table 'obj_uuid_table', created 11/10/2025 05:02:24.884 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/10/2025 05:02:25.784 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/10/2025 05:02:26.147 7f2f9524e0f0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:02:26.148 7f2f9524e0f0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:02:26.149 7f2f9524e0f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:02:26.149 7f2f9524e0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:02:26.149 7f2f9524e0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 11/10/2025 05:02:26.149 7f2f9524e0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:02:26.149 7f2f9524e0f0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:02:26.150 7f2f9524e0f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:02:26.150 7f2f9524e0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:26.150 7f2f9524e0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 11/10/2025 05:02:26.150 7f2f9524e0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:26.151 7f2f9524e0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:26.151 7f2f9524e0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:02:26.151 7f2f9524e0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:26.711 7f2f9deda748 [contrail-device-manager] [WARNING]: table 'obj_uuid_table' fixed with properties {'gc_grace_seconds': 864000} 11/10/2025 05:02:26.721 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/10/2025 05:02:27.990 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/10/2025 05:02:29.621 7f2f9deda748 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table', created 11/10/2025 05:02:29.627 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/10/2025 05:02:30.150 7f2f95346f60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:02:30.151 7f2f95346f60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:02:30.151 7f2f95346f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:02:30.151 7f2f95346f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:02:30.151 7f2f95346f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 11/10/2025 05:02:30.151 7f2f95346f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:02:30.152 7f2f95346f60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:02:30.152 7f2f95346f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:02:30.152 7f2f95346f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:30.152 7f2f95346f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 11/10/2025 05:02:30.152 7f2f95346f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:30.152 7f2f95346f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:30.152 7f2f95346f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:02:30.152 7f2f95346f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:30.853 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/10/2025 05:02:31.771 7f2f9deda748 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table' fixed with properties {'gc_grace_seconds': 864000} 11/10/2025 05:02:31.776 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/10/2025 05:02:34.152 7f2f949dfcc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:02:34.153 7f2f949dfcc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:02:34.154 7f2f949dfcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:02:34.154 7f2f949dfcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:02:34.154 7f2f949dfcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 11/10/2025 05:02:34.154 7f2f949dfcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:02:34.154 7f2f949dfcc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:02:34.154 7f2f949dfcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:02:34.154 7f2f949dfcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:34.155 7f2f949dfcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 11/10/2025 05:02:34.155 7f2f949dfcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:34.155 7f2f949dfcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:34.155 7f2f949dfcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:02:34.155 7f2f949dfcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:34.162 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/10/2025 05:02:34.164 7f2f9deda748 [contrail-device-manager] [WARNING]: table 'obj_shared_table', created 11/10/2025 05:02:34.174 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/10/2025 05:02:36.015 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/10/2025 05:02:36.933 7f2f9deda748 [contrail-device-manager] [WARNING]: table 'obj_shared_table' fixed with properties {'gc_grace_seconds': 864000} 11/10/2025 05:02:36.938 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/10/2025 05:02:36.939 7f2f9ab5c048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:36.939 7f2f9ab5c048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:02:36.940 7f2f9ab5c048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:36.954 7f2f9deda748 [contrail-device-manager] [ERROR]: Error in db walk read 'type' 11/10/2025 05:02:36.956 7f2f949e2f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:36.957 7f2f949e2f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Zookeeper:Zookeeper[] connection down >>, ] >> 11/10/2025 05:02:36.957 7f2f949e2f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:36.973 7f2f949e2198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:36.974 7f2f949e2198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:02:36.974 7f2f949e2198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:36.974 7f2f9deda748 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 11/10/2025 05:02:36.974 7f2f9deda748 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 11/10/2025 05:02:36.974 7f2fa3cda630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/10/2025 05:02:36.974 7f2f9deda748 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4678-3. 11/10/2025 05:02:36.986 7f2fa3cda630 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/10/2025 05:02:37.263 7f2fa3cda630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4678-3.'] 11/10/2025 05:02:37.288 7f2fa3cda630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/10/2025 05:02:37.295 7f2fa3cda630 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/10/2025 05:02:37.341 7f2fa3cda630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4678-3.'] 11/10/2025 05:02:37.490 7f2f9deda748 [contrail-device-manager] [DEBUG]: Fetching all containers 11/10/2025 05:02:37.553 7f2f9deda748 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 11/10/2025 05:02:37.866 7f2f9deda748 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4678-3..config_queue 11/10/2025 05:02:37.867 7f2f9deda748 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4678-3..tftp_queue 11/10/2025 05:02:38.154 7f2f94922780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:02:38.155 7f2f94922780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:02:38.155 7f2f94922780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:02:38.155 7f2f94922780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:02:38.156 7f2f94922780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:02:38.156 7f2f94922780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:02:38.156 7f2f94922780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:02:38.156 7f2f94922780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:02:38.156 7f2f94922780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:38.156 7f2f94922780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:02:38.156 7f2f94922780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:38.156 7f2f94922780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:38.156 7f2f94922780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:02:38.156 7f2f94922780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:38.363 7f2fa3cda630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/10/2025 05:02:38.367 7f2f9deda748 [contrail-device-manager] [WARNING]: Process 140 prepared to run in Full mode... 11/10/2025 05:02:38.367 7f2f9deda748 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 11/10/2025 05:02:38.367 7f2f9deda748 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 11/10/2025 05:02:38.370 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page 11/10/2025 05:02:38.370 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 11/10/2025 05:02:38.371 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page 11/10/2025 05:02:38.371 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 11/10/2025 05:02:38.373 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 11/10/2025 05:02:38.373 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 11/10/2025 05:02:38.377 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page 11/10/2025 05:02:38.377 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 11/10/2025 05:02:38.377 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 11/10/2025 05:02:38.377 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 11/10/2025 05:02:38.377 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 11/10/2025 05:02:38.377 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 11/10/2025 05:02:38.377 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 11/10/2025 05:02:38.377 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 11/10/2025 05:02:38.377 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 11/10/2025 05:02:38.377 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 11/10/2025 05:02:38.378 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 11/10/2025 05:02:38.378 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 11/10/2025 05:02:38.378 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 11/10/2025 05:02:38.379 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 11/10/2025 05:02:38.379 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 11/10/2025 05:02:38.379 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 11/10/2025 05:02:38.379 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 11/10/2025 05:02:38.380 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 11/10/2025 05:02:38.380 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 11/10/2025 05:02:38.380 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 11/10/2025 05:02:38.381 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page 11/10/2025 05:02:38.381 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 11/10/2025 05:02:38.381 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 11/10/2025 05:02:38.381 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 11/10/2025 05:02:38.381 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 11/10/2025 05:02:38.381 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 11/10/2025 05:02:38.382 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page 11/10/2025 05:02:38.382 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 11/10/2025 05:02:38.382 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 11/10/2025 05:02:38.382 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 11/10/2025 05:02:38.382 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 11/10/2025 05:02:38.382 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 11/10/2025 05:02:38.382 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 11/10/2025 05:02:38.382 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 11/10/2025 05:02:38.382 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 11/10/2025 05:02:38.382 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 11/10/2025 05:02:38.383 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 11/10/2025 05:02:38.383 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 11/10/2025 05:02:38.383 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 11/10/2025 05:02:38.383 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 11/10/2025 05:02:38.383 7f2f9deda748 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 11/10/2025 05:02:38.386 7f2f9deda748 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 11/10/2025 05:02:38.400 7f2f9498e828 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 11/10/2025 05:02:38.400 7f2f9498e828 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 11/10/2025 05:02:38.451 7f2f94159f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:38.452 7f2f94159f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 11/10/2025 05:02:38.452 7f2f94159f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:38.455 7f2fa3cda630 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/10/2025 05:02:38.467 7f2f94043a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:38.467 7f2f94043a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 11/10/2025 05:02:38.468 7f2f94043a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:38.547 7f2fa3cda630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4678-3.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4678-3..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4678-3..tftp_queue', 'device_manager_ztp.ztp_queue'] 11/10/2025 05:02:38.566 7f2fa3cda630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/10/2025 05:02:38.571 7f2fa3cda630 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/10/2025 05:02:38.610 7f2fa3cda630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4678-3.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4678-3..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4678-3..tftp_queue', 'device_manager_ztp.ztp_queue'] 11/10/2025 05:02:42.156 7f2f8ffb26d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:02:42.157 7f2f8ffb26d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:02:42.157 7f2f8ffb26d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:02:42.157 7f2f8ffb26d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:02:42.157 7f2f8ffb26d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 11/10/2025 05:02:42.157 7f2f8ffb26d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:02:42.157 7f2f8ffb26d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:02:42.158 7f2f8ffb26d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:02:42.158 7f2f8ffb26d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:42.158 7f2f8ffb26d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 11/10/2025 05:02:42.158 7f2f8ffb26d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:42.158 7f2f8ffb26d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:42.158 7f2f8ffb26d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:02:42.159 7f2f8ffb26d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:44.629 7f2f8ffafd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:44.629 7f2f8ffafd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:02:44.629 7f2f8ffafd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:46.158 7f2f8ff2a6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:02:46.160 7f2f8ff2a6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:02:46.160 7f2f8ff2a6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:02:46.160 7f2f8ff2a6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:02:46.160 7f2f8ff2a6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:02:46.160 7f2f8ff2a6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:02:46.160 7f2f8ff2a6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:02:46.160 7f2f8ff2a6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:02:46.161 7f2f8ff2a6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:46.161 7f2f8ff2a6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:02:46.161 7f2f8ff2a6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:46.161 7f2f8ff2a6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:46.161 7f2f8ff2a6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:02:46.161 7f2f8ff2a6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:50.160 7f2f94983588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:02:50.162 7f2f94983588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:02:50.162 7f2f94983588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:02:50.162 7f2f94983588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:02:50.162 7f2f94983588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:02:50.162 7f2f94983588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:02:50.163 7f2f94983588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:02:50.163 7f2f94983588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:02:50.163 7f2f94983588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:50.163 7f2f94983588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:02:50.163 7f2f94983588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:50.163 7f2f94983588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:50.163 7f2f94983588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:02:50.163 7f2f94983588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:54.163 7f2f8fedb828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:02:54.164 7f2f8fedb828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:02:54.165 7f2f8fedb828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:02:54.165 7f2f8fedb828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:02:54.165 7f2f8fedb828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:02:54.165 7f2f8fedb828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:02:54.165 7f2f8fedb828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:02:54.166 7f2f8fedb828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:02:54.166 7f2f8fedb828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:54.166 7f2f8fedb828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:02:54.166 7f2f8fedb828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:54.166 7f2f8fedb828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:54.166 7f2f8fedb828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:02:54.166 7f2f8fedb828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:58.165 7f2f8ff34048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:02:58.167 7f2f8ff34048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:02:58.167 7f2f8ff34048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:02:58.167 7f2f8ff34048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:02:58.167 7f2f8ff34048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:02:58.167 7f2f8ff34048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:02:58.168 7f2f8ff34048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:02:58.168 7f2f8ff34048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:02:58.168 7f2f8ff34048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:58.168 7f2f8ff34048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:02:58.168 7f2f8ff34048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:58.168 7f2f8ff34048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:02:58.168 7f2f8ff34048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:02:58.168 7f2f8ff34048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:02.168 7f2f8ff34048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:03:02.170 7f2f8ff34048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:03:02.170 7f2f8ff34048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:03:02.170 7f2f8ff34048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:02.170 7f2f8ff34048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:03:02.170 7f2f8ff34048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:02.170 7f2f8ff34048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:03:02.171 7f2f8ff34048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:03:02.171 7f2f8ff34048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:02.171 7f2f8ff34048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:03:02.171 7f2f8ff34048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:02.171 7f2f8ff34048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:02.171 7f2f8ff34048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:03:02.171 7f2f8ff34048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:06.171 7f2f8ff34048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:03:06.173 7f2f8ff34048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:03:06.173 7f2f8ff34048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:03:06.173 7f2f8ff34048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:06.173 7f2f8ff34048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:03:06.173 7f2f8ff34048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:06.173 7f2f8ff34048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:03:06.173 7f2f8ff34048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:03:06.174 7f2f8ff34048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:06.174 7f2f8ff34048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:03:06.174 7f2f8ff34048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:06.174 7f2f8ff34048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:06.174 7f2f8ff34048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:03:06.174 7f2f8ff34048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:10.177 7f2f9534acc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:03:10.180 7f2f9534acc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:03:10.180 7f2f9534acc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:03:10.180 7f2f9534acc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:10.180 7f2f9534acc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:03:10.180 7f2f9534acc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:10.180 7f2f9534acc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:03:10.181 7f2f9534acc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:03:10.181 7f2f9534acc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:10.181 7f2f9534acc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:03:10.181 7f2f9534acc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:10.181 7f2f9534acc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:10.181 7f2f9534acc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:03:10.181 7f2f9534acc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:14.180 7f2f940bc978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:03:14.181 7f2f940bc978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:03:14.182 7f2f940bc978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:03:14.182 7f2f940bc978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:14.182 7f2f940bc978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:03:14.182 7f2f940bc978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:14.182 7f2f940bc978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:03:14.182 7f2f940bc978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:03:14.182 7f2f940bc978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:14.182 7f2f940bc978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:03:14.183 7f2f940bc978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:14.183 7f2f940bc978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:14.183 7f2f940bc978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:03:14.183 7f2f940bc978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:18.182 7f2f940bc8d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:03:18.186 7f2f940bc8d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:03:18.186 7f2f940bc8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:03:18.186 7f2f940bc8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:18.187 7f2f940bc8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:03:18.187 7f2f940bc8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:18.187 7f2f940bc8d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:03:18.187 7f2f940bc8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:03:18.188 7f2f940bc8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:18.188 7f2f940bc8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:03:18.188 7f2f940bc8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:18.188 7f2f940bc8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:18.188 7f2f940bc8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:03:18.188 7f2f940bc8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:22.187 7f2f8fedbe10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:03:22.189 7f2f8fedbe10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:03:22.189 7f2f8fedbe10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:03:22.189 7f2f8fedbe10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:22.189 7f2f8fedbe10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:03:22.189 7f2f8fedbe10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:22.189 7f2f8fedbe10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:03:22.189 7f2f8fedbe10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:03:22.189 7f2f8fedbe10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:22.190 7f2f8fedbe10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:03:22.190 7f2f8fedbe10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:22.190 7f2f8fedbe10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:22.190 7f2f8fedbe10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:03:22.190 7f2f8fedbe10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:26.190 7f2f940bc978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:03:26.191 7f2f940bc978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:03:26.191 7f2f940bc978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:03:26.192 7f2f940bc978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:26.192 7f2f940bc978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:03:26.192 7f2f940bc978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:26.192 7f2f940bc978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:03:26.192 7f2f940bc978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:03:26.192 7f2f940bc978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:26.192 7f2f940bc978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:03:26.193 7f2f940bc978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:26.193 7f2f940bc978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:26.193 7f2f940bc978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:03:26.193 7f2f940bc978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:30.192 7f2f95272eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:03:30.193 7f2f95272eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:03:30.193 7f2f95272eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:03:30.193 7f2f95272eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:30.194 7f2f95272eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:03:30.194 7f2f95272eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:30.194 7f2f95272eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:03:30.194 7f2f95272eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:03:30.194 7f2f95272eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:30.194 7f2f95272eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:03:30.194 7f2f95272eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:30.194 7f2f95272eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:30.194 7f2f95272eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:03:30.194 7f2f95272eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:34.195 7f2f95337c18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:03:34.196 7f2f95337c18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:03:34.196 7f2f95337c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:03:34.197 7f2f95337c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:34.197 7f2f95337c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:03:34.197 7f2f95337c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:34.197 7f2f95337c18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:03:34.197 7f2f95337c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:03:34.197 7f2f95337c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:34.197 7f2f95337c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:03:34.197 7f2f95337c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:34.198 7f2f95337c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:34.198 7f2f95337c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:03:34.198 7f2f95337c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:38.197 7f2f8fedb630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:03:38.198 7f2f8fedb630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:03:38.198 7f2f8fedb630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:03:38.198 7f2f8fedb630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:38.199 7f2f8fedb630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:03:38.199 7f2f8fedb630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:38.199 7f2f8fedb630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:03:38.199 7f2f8fedb630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:03:38.199 7f2f8fedb630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:38.199 7f2f8fedb630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:03:38.199 7f2f8fedb630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:38.199 7f2f8fedb630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:38.199 7f2f8fedb630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:03:38.200 7f2f8fedb630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:42.199 7f2f8fedb978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:03:42.201 7f2f8fedb978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:03:42.201 7f2f8fedb978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:03:42.201 7f2f8fedb978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:42.202 7f2f8fedb978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:03:42.202 7f2f8fedb978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:42.203 7f2f8fedb978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:03:42.203 7f2f8fedb978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:03:42.203 7f2f8fedb978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:42.203 7f2f8fedb978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:03:42.204 7f2f8fedb978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:42.204 7f2f8fedb978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:42.204 7f2f8fedb978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:03:42.204 7f2f8fedb978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:43.281 7f2f9deda748 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 11/10/2025 05:03:43.281 7f2f9deda748 [contrail-device-manager] [WARNING]: initializing cassandra cluster 11/10/2025 05:03:43.282 7f2f9deda748 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 11/10/2025 05:03:43.282 7f2f8ffe5588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:43.282 7f2f8ffe5588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 11/10/2025 05:03:43.282 7f2f8ffe5588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:43.283 7f2f9deda748 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 11/10/2025 05:03:43.320 7f2f9deda748 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 11/10/2025 05:03:43.321 7f2f940f58d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:43.322 7f2f940f58d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Initializing description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 11/10/2025 05:03:43.322 7f2f940f58d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:43.332 7f2f9deda748 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 11/10/2025 05:03:43.332 7f2f940f56d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:43.333 7f2f940f56d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 11/10/2025 05:03:43.333 7f2f940f56d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:43.355 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/10/2025 05:03:43.949 7f2f9deda748 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace', created 11/10/2025 05:03:43.953 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/10/2025 05:03:43.960 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/10/2025 05:03:44.645 7f2f9deda748 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None} 11/10/2025 05:03:44.651 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/10/2025 05:03:44.651 7f2f9deda748 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 11/10/2025 05:03:44.716 7f2f9deda748 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 11/10/2025 05:03:44.723 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/10/2025 05:03:45.279 7f2f9deda748 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table', created 11/10/2025 05:03:45.284 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/10/2025 05:03:45.292 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/10/2025 05:03:46.017 7f2f9deda748 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table' fixed with properties {'gc_grace_seconds': 864000} 11/10/2025 05:03:46.022 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/10/2025 05:03:46.032 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/10/2025 05:03:46.201 7f2f8feee780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:03:46.203 7f2f8feee780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:03:46.203 7f2f8feee780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:03:46.203 7f2f8feee780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:46.204 7f2f8feee780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 11/10/2025 05:03:46.204 7f2f8feee780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:46.204 7f2f8feee780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:03:46.204 7f2f8feee780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:03:46.204 7f2f8feee780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:46.204 7f2f8feee780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 11/10/2025 05:03:46.205 7f2f8feee780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:46.205 7f2f8feee780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:46.205 7f2f8feee780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:03:46.205 7f2f8feee780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:47.779 7f2f9deda748 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table', created 11/10/2025 05:03:47.786 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/10/2025 05:03:47.792 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/10/2025 05:03:48.294 7f2f9deda748 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table' fixed with properties {'gc_grace_seconds': 864000} 11/10/2025 05:03:48.299 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/10/2025 05:03:48.309 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/10/2025 05:03:49.095 7f2f9deda748 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table', created 11/10/2025 05:03:49.099 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/10/2025 05:03:49.106 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/10/2025 05:03:50.204 7f2f8fea00f0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:03:50.205 7f2f8fea00f0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:03:50.206 7f2f8fea00f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:03:50.206 7f2f8fea00f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:50.206 7f2f8fea00f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 11/10/2025 05:03:50.206 7f2f8fea00f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:50.206 7f2f8fea00f0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:03:50.206 7f2f8fea00f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:03:50.206 7f2f8fea00f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:50.207 7f2f8fea00f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 11/10/2025 05:03:50.207 7f2f8fea00f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:50.207 7f2f8fea00f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:50.207 7f2f8fea00f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:03:50.207 7f2f8fea00f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:50.211 7f2f9deda748 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table' fixed with properties {'gc_grace_seconds': 864000} 11/10/2025 05:03:50.216 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/10/2025 05:03:50.221 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/10/2025 05:03:50.975 7f2f9deda748 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table', created 11/10/2025 05:03:50.979 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/10/2025 05:03:50.985 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/10/2025 05:03:51.956 7f2f9deda748 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table' fixed with properties {'gc_grace_seconds': 864000} 11/10/2025 05:03:51.961 7f2f9deda748 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/10/2025 05:03:51.962 7f2f940f5ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:51.962 7f2f940f5ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:03:51.962 7f2f940f5ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:52.024 7f2f9deda748 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//', 'pyamqp://guest:guest@10.0.0.33:5673//'] 11/10/2025 05:03:52.025 7f2f9deda748 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 11/10/2025 05:03:52.025 7f2f8fe9dac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:52.026 7f2f8fe9dac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Initializing description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 11/10/2025 05:03:52.026 7f2f8fe9dac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:52.026 7f2f8fe9dac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:52.026 7f2f8fe9dac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 11/10/2025 05:03:52.026 7f2f8fe9dac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:52.035 7f2f9deda748 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 11/10/2025 05:03:52.036 7f2f8fe9d978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:52.036 7f2f8fe9d978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:03:52.036 7f2f8fe9d978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:52.086 7f2f9deda748 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 11/10/2025 05:03:52.087 7f2f8fe96978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:52.087 7f2f8fe96978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 11/10/2025 05:03:52.087 7f2f8fe96978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:52.094 7f2f9deda748 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 11/10/2025 05:03:52.095 7f2f8fea4c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:52.095 7f2f8fea4c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:03:52.095 7f2f8fea4c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:52.096 7f2f8fe966d8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 11/10/2025 05:03:52.097 7f2f8fea48d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:52.097 7f2f8fea48d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 11/10/2025 05:03:52.097 7f2f8fea48d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:52.102 7f2f8fe966d8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 11/10/2025 05:03:52.102 7f2f8fe9fe10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:52.102 7f2f8fe9fe10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:03:52.102 7f2f8fe9fe10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:54.207 7f2f8fe9e438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:03:54.208 7f2f8fe9e438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:03:54.208 7f2f8fe9e438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:03:54.208 7f2f8fe9e438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:54.208 7f2f8fe9e438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:03:54.208 7f2f8fe9e438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:54.208 7f2f8fe9e438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:03:54.209 7f2f8fe9e438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:03:54.209 7f2f8fe9e438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:54.209 7f2f8fe9e438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:03:54.209 7f2f8fe9e438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:54.209 7f2f8fe9e438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:54.209 7f2f8fe9e438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:03:54.209 7f2f8fe9e438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:58.209 7f2f8fe9e438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:03:58.211 7f2f8fe9e438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:03:58.211 7f2f8fe9e438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:03:58.211 7f2f8fe9e438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:58.211 7f2f8fe9e438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:03:58.212 7f2f8fe9e438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:03:58.212 7f2f8fe9e438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:03:58.212 7f2f8fe9e438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:03:58.212 7f2f8fe9e438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:58.212 7f2f8fe9e438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:03:58.213 7f2f8fe9e438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:58.213 7f2f8fe9e438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:03:58.213 7f2f8fe9e438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:03:58.213 7f2f8fe9e438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:02.212 7f2f8fe9e438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:04:02.214 7f2f8fe9e438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:04:02.214 7f2f8fe9e438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:04:02.214 7f2f8fe9e438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:04:02.214 7f2f8fe9e438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:04:02.214 7f2f8fe9e438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:04:02.214 7f2f8fe9e438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:04:02.215 7f2f8fe9e438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:04:02.215 7f2f8fe9e438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:02.215 7f2f8fe9e438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:04:02.215 7f2f8fe9e438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:02.215 7f2f8fe9e438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:02.215 7f2f8fe9e438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:04:02.215 7f2f8fe9e438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:06.215 7f2f8fe9e438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:04:06.216 7f2f8fe9e438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:04:06.216 7f2f8fe9e438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:04:06.217 7f2f8fe9e438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:04:06.217 7f2f8fe9e438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:04:06.217 7f2f8fe9e438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:04:06.217 7f2f8fe9e438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:04:06.217 7f2f8fe9e438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:04:06.217 7f2f8fe9e438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:06.217 7f2f8fe9e438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:04:06.217 7f2f8fe9e438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:06.218 7f2f8fe9e438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:06.218 7f2f8fe9e438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:04:06.218 7f2f8fe9e438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:10.218 7f2f8fe9e438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:04:10.222 7f2f8fe9e438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:04:10.222 7f2f8fe9e438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:04:10.222 7f2f8fe9e438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:04:10.222 7f2f8fe9e438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:04:10.222 7f2f8fe9e438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:04:10.223 7f2f8fe9e438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:04:10.223 7f2f8fe9e438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:04:10.223 7f2f8fe9e438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:10.223 7f2f8fe9e438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:04:10.223 7f2f8fe9e438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:10.224 7f2f8fe9e438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:10.224 7f2f8fe9e438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:04:10.224 7f2f8fe9e438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:11.729 7f2f8fe96e10 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4678-3.'], 'obj_dict': {'database_node_ip_address': '10.0.0.33', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4678-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4678-3.'], 'id_perms': {'created': '2025-11-10T05:04:11.706953', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-10T05:04:11.706953', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13396218647791627896, 'uuid_mslong': 2789787750673960772}}, 'parent_type': 'global-system-config', 'parent_uuid': 'fe3aad61-f1d2-4bf1-9786-0643d55a11b7', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '26b7512d-a5a8-4f44-b9e8-ede24b156e78'}, 'oper': 'CREATE', 'request-id': 'req-7c67e459-89cf-4799-a4bb-2895af134f1f', 'type': 'database_node', 'uuid': '26b7512d-a5a8-4f44-b9e8-ede24b156e78'} 11/10/2025 05:04:12.221 7f2f8fe96e10 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4678-2.'], 'obj_dict': {'database_node_ip_address': '10.0.0.50', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4678-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4678-2.'], 'id_perms': {'created': '2025-11-10T05:04:12.204515', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-10T05:04:12.204515', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9375417163614331821, 'uuid_mslong': 11595803371094884812}}, 'parent_type': 'global-system-config', 'parent_uuid': 'fe3aad61-f1d2-4bf1-9786-0643d55a11b7', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'a0ec91ba-9363-41cc-821c-2c3cba5523ad'}, 'oper': 'CREATE', 'request-id': 'req-ee34b0b0-2581-436e-a111-63acdb64eedc', 'type': 'database_node', 'uuid': 'a0ec91ba-9363-41cc-821c-2c3cba5523ad'} 11/10/2025 05:04:12.311 7f2f8fe96e10 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4678-1.'], 'obj_dict': {'database_node_ip_address': '10.0.0.38', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4678-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4678-1.'], 'id_perms': {'created': '2025-11-10T05:04:12.291338', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-10T05:04:12.291338', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13484773820189388961, 'uuid_mslong': 14001676447274848024}}, 'parent_type': 'global-system-config', 'parent_uuid': 'fe3aad61-f1d2-4bf1-9786-0643d55a11b7', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'c24ff28b-73c5-4f18-bb23-8a5814e030a1'}, 'oper': 'CREATE', 'request-id': 'req-0da7bd7f-17d6-43ea-936a-f22b922f4d06', 'type': 'database_node', 'uuid': 'c24ff28b-73c5-4f18-bb23-8a5814e030a1'} 11/10/2025 05:04:14.223 7f2f8fea4780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:04:14.224 7f2f8fea4780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:04:14.224 7f2f8fea4780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:04:14.224 7f2f8fea4780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:04:14.225 7f2f8fea4780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:04:14.225 7f2f8fea4780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:04:14.225 7f2f8fea4780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:04:14.225 7f2f8fea4780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:04:14.225 7f2f8fea4780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:14.225 7f2f8fea4780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:04:14.225 7f2f8fea4780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:14.225 7f2f8fea4780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:14.225 7f2f8fea4780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:04:14.226 7f2f8fea4780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:18.225 7f2f8fea4780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:04:18.226 7f2f8fea4780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:04:18.226 7f2f8fea4780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:04:18.227 7f2f8fea4780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:04:18.227 7f2f8fea4780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:04:18.227 7f2f8fea4780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:04:18.227 7f2f8fea4780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:04:18.227 7f2f8fea4780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:04:18.227 7f2f8fea4780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:18.227 7f2f8fea4780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:04:18.227 7f2f8fea4780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:18.227 7f2f8fea4780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:18.228 7f2f8fea4780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:04:18.228 7f2f8fea4780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:22.227 7f2f8fea4780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:04:22.229 7f2f8fea4780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:04:22.229 7f2f8fea4780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:04:22.229 7f2f8fea4780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:04:22.229 7f2f8fea4780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:04:22.229 7f2f8fea4780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:04:22.229 7f2f8fea4780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:04:22.229 7f2f8fea4780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:04:22.229 7f2f8fea4780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:22.230 7f2f8fea4780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:04:22.230 7f2f8fea4780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:22.230 7f2f8fea4780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:22.230 7f2f8fea4780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:04:22.230 7f2f8fea4780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:26.230 7f2f952774e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:04:26.232 7f2f952774e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:04:26.232 7f2f952774e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:04:26.232 7f2f952774e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:04:26.232 7f2f952774e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:04:26.232 7f2f952774e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:04:26.232 7f2f952774e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:04:26.233 7f2f952774e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:04:26.233 7f2f952774e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:26.233 7f2f952774e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:04:26.233 7f2f952774e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:26.233 7f2f952774e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:26.233 7f2f952774e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:04:26.233 7f2f952774e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:30.233 7f2f952774e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:04:30.234 7f2f952774e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:04:30.234 7f2f952774e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:04:30.234 7f2f952774e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:04:30.234 7f2f952774e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:04:30.234 7f2f952774e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:04:30.234 7f2f952774e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:04:30.235 7f2f952774e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:04:30.235 7f2f952774e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:30.235 7f2f952774e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:04:30.235 7f2f952774e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:30.235 7f2f952774e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:30.235 7f2f952774e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:04:30.235 7f2f952774e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:34.235 7f2f952774e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:04:34.237 7f2f952774e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:04:34.237 7f2f952774e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:04:34.237 7f2f952774e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:04:34.237 7f2f952774e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:04:34.237 7f2f952774e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:04:34.237 7f2f952774e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:04:34.237 7f2f952774e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:04:34.238 7f2f952774e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:34.238 7f2f952774e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:04:34.238 7f2f952774e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:34.238 7f2f952774e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:34.238 7f2f952774e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:04:34.238 7f2f952774e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:38.238 7f2f952774e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:04:38.239 7f2f952774e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 05:04:38.239 7f2f952774e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:04:38.239 7f2f952774e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:04:38.240 7f2f952774e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:04:38.240 7f2f952774e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:04:38.240 7f2f952774e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 05:04:38.240 7f2f952774e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 05:04:38.240 7f2f952774e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:38.240 7f2f952774e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 05:04:38.240 7f2f952774e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:38.240 7f2f952774e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:38.240 7f2f952774e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1762750942146295 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 05:04:38.240 7f2f952774e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 05:04:41.982 7f2f8fe96e10 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4678-2.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.50', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4678-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4678-2.'], 'id_perms': {'created': '2025-11-10T05:04:41.962299', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-10T05:04:41.962299', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11195522418746949016, 'uuid_mslong': 7646266474245999043}}, 'parent_type': 'global-system-config', 'parent_uuid': 'fe3aad61-f1d2-4bf1-9786-0643d55a11b7', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '6a1cfed8-bf69-49c3-9b5e-7c52cc557d98'}, 'oper': 'CREATE', 'request-id': 'req-d781e3ab-f69b-463d-a531-76a188ace8be', 'type': 'analytics_node', 'uuid': '6a1cfed8-bf69-49c3-9b5e-7c52cc557d98'} 11/10/2025 05:04:41.997 7f2f8fe96e10 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4678-1.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.38', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4678-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4678-1.'], 'id_perms': {'created': '2025-11-10T05:04:41.978060', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-10T05:04:41.978060', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11309205638906460435, 'uuid_mslong': 5177877226430416299}}, 'parent_type': 'global-system-config', 'parent_uuid': 'fe3aad61-f1d2-4bf1-9786-0643d55a11b7', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '47db83e7-63f8-45ab-9cf2-5e9c1edd3913'}, 'oper': 'CREATE', 'request-id': 'req-a71214ad-0070-4c43-b8d4-5daef9ded65d', 'type': 'analytics_node', 'uuid': '47db83e7-63f8-45ab-9cf2-5e9c1edd3913'} 11/10/2025 05:04:42.008 7f2f8fe96e10 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4678-3.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.33', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4678-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4678-3.'], 'id_perms': {'created': '2025-11-10T05:04:41.989592', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-10T05:04:41.989592', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13541271171400251839, 'uuid_mslong': 448265891486519294}}, 'parent_type': 'global-system-config', 'parent_uuid': 'fe3aad61-f1d2-4bf1-9786-0643d55a11b7', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '06388f74-0afa-4ffe-bbec-4262ba8cb9bf'}, 'oper': 'CREATE', 'request-id': 'req-3a5264bb-3d43-409d-8ada-838379d985dc', 'type': 'analytics_node', 'uuid': '06388f74-0afa-4ffe-bbec-4262ba8cb9bf'} 11/10/2025 05:04:42.240 7f2f8fe8cf60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 05:04:42.241 7f2f8fe8cf60 [contrail-device-manager] [INFO]: Session Event: TCP Connected 11/10/2025 05:04:42.241 7f2f8fe8cf60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 05:04:42.241 7f2f8fe8cf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:04:42.242 7f2f8fe8cf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4678-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.50:5673, 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.50:9041, 10.0.0.33:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.50:2181, 10.0.0.33:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.38,10.0.0.50,10.0.0.33:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 05:04:42.242 7f2f8fe8cf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 05:04:42.242 7f2f8fe8cf60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect] 11/10/2025 05:04:42.242 7f2f8fe8cf60 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19 11/10/2025 05:04:42.242 7f2f8fe8cf60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 11/10/2025 05:04:42.242 7f2f8fe8cf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/10/2025 05:04:42.242 7f2f8fe8cf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/10/2025 05:04:42.243 7f2f8fe9d438 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 11/10/2025 05:04:42.244 7f2f8fe9d438 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 11/10/2025 05:04:54.685 7f2f8fe96438 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001016 11/10/2025 05:04:54.686 7f2f8fe96438 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 11/10/2025 05:04:54.686 7f2f9ce1f828 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 11/10/2025 05:04:54.686 7f2f9ce1f828 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 0 11/10/2025 05:04:54.686 7f2f9ce1f828 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 11/10/2025 05:04:54.686 7f2f9ce1f828 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/10/2025 05:04:54.687 7f2f9ce1f828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 11/10/2025 05:04:54.687 7f2f8fe978d0 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 11/10/2025 05:04:54.688 7f2f8fe978d0 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 11/10/2025 05:04:57.448 7f2f8fe96e10 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4678-2.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.50', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4678-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4678-2.'], 'id_perms': {'created': '2025-11-10T05:04:57.429073', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-10T05:04:57.429073', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12218703200933907737, 'uuid_mslong': 1192295717579476518}}, 'parent_type': 'global-system-config', 'parent_uuid': 'fe3aad61-f1d2-4bf1-9786-0643d55a11b7', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '108be2a3-598b-4e26-a991-8dd2e3980d19'}, 'oper': 'CREATE', 'request-id': 'req-5c53668f-3ccb-4ac3-ab9b-938b02e634a4', 'type': 'analytics_alarm_node', 'uuid': '108be2a3-598b-4e26-a991-8dd2e3980d19'} 11/10/2025 05:04:58.312 7f2f8fe96e10 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4678-3.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.33', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4678-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4678-3.'], 'id_perms': {'created': '2025-11-10T05:04:58.295758', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-10T05:04:58.295758', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10022719447341197990, 'uuid_mslong': 2524325220048519521}}, 'parent_type': 'global-system-config', 'parent_uuid': 'fe3aad61-f1d2-4bf1-9786-0643d55a11b7', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '23083469-c68d-4161-8b17-da3c162beaa6'}, 'oper': 'CREATE', 'request-id': 'req-4b94af5e-a49a-4cfb-b943-141aed028feb', 'type': 'analytics_alarm_node', 'uuid': '23083469-c68d-4161-8b17-da3c162beaa6'} 11/10/2025 05:04:58.394 7f2f8fe96e10 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4678-1.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.38', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4678-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4678-1.'], 'id_perms': {'created': '2025-11-10T05:04:58.339906', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-10T05:04:58.339906', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10292862632624421927, 'uuid_mslong': 12478100422655951077}}, 'parent_type': 'global-system-config', 'parent_uuid': 'fe3aad61-f1d2-4bf1-9786-0643d55a11b7', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'ad2b1e38-fc0f-40e5-8ed7-9807cb787c27'}, 'oper': 'CREATE', 'request-id': 'req-224b7f7a-644c-4cdf-bbfc-b9c321e1d51a', 'type': 'analytics_alarm_node', 'uuid': 'ad2b1e38-fc0f-40e5-8ed7-9807cb787c27'} 11/10/2025 05:05:04.676 7f2f8fe96438 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001296 11/10/2025 05:05:04.676 7f2f8fe96438 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 11/10/2025 05:05:04.676 7f2f8fedb780 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 11/10/2025 05:05:04.677 7f2f8fedb780 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 2 11/10/2025 05:05:04.677 7f2f8fedb780 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 11/10/2025 05:05:04.677 7f2f8fedb780 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/10/2025 05:05:09.101 7f2f8fe96e10 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4678-2.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.50', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4678-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4678-2.'], 'id_perms': {'created': '2025-11-10T05:05:09.074492', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-10T05:05:09.074492', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10311445032715267131, 'uuid_mslong': 9562544353342605045}}, 'parent_type': 'global-system-config', 'parent_uuid': 'fe3aad61-f1d2-4bf1-9786-0643d55a11b7', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '84b4fb6c-c1e7-46f5-8f19-9c9ff7f5ac3b'}, 'oper': 'CREATE', 'request-id': 'req-74b759ef-7c79-4067-95a3-a2ebe705a7be', 'type': 'analytics_snmp_node', 'uuid': '84b4fb6c-c1e7-46f5-8f19-9c9ff7f5ac3b'} 11/10/2025 05:05:09.122 7f2f8fe96e10 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4678-1.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.38', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4678-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4678-1.'], 'id_perms': {'created': '2025-11-10T05:05:09.085755', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-10T05:05:09.085755', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11829692279491043831, 'uuid_mslong': 16750978531345449400}}, 'parent_type': 'global-system-config', 'parent_uuid': 'fe3aad61-f1d2-4bf1-9786-0643d55a11b7', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'e8776e39-490a-41b8-a42b-8273a93ea1f7'}, 'oper': 'CREATE', 'request-id': 'req-1eec4551-f2b8-471e-9d28-84a108284bf4', 'type': 'analytics_snmp_node', 'uuid': 'e8776e39-490a-41b8-a42b-8273a93ea1f7'} 11/10/2025 05:05:09.185 7f2f8fe96e10 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4678-3.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.33', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4678-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4678-3.'], 'id_perms': {'created': '2025-11-10T05:05:09.167815', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-10T05:05:09.167815', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12056759684039826000, 'uuid_mslong': 13363772519336069894}}, 'parent_type': 'global-system-config', 'parent_uuid': 'fe3aad61-f1d2-4bf1-9786-0643d55a11b7', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'b975a84d-5227-4b06-a752-370d9d71da50'}, 'oper': 'CREATE', 'request-id': 'req-1fb66d08-6f0b-4012-8ccf-2457e2f7ec6b', 'type': 'analytics_snmp_node', 'uuid': 'b975a84d-5227-4b06-a752-370d9d71da50'} 11/10/2025 05:06:57.072 7f2f8fe96e10 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-22f91885-4661-4f11-82d4-5e3efc8b3f4f', 'type': 'global_system_config', 'uuid': 'fe3aad61-f1d2-4bf1-9786-0643d55a11b7'} 11/10/2025 05:06:57.150 7f2f8fe96e10 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-57c56cd2-800e-4853-9e53-b02f46187d15', 'type': 'global_system_config', 'uuid': 'fe3aad61-f1d2-4bf1-9786-0643d55a11b7'} 11/10/2025 05:06:57.244 7f2f8fe96e10 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4678-1'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-4678-1', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4678-1'], 'id_perms': {'created': '2025-11-10T05:06:57.227753', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-10T05:06:57.227753', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9660888044673252065, 'uuid_mslong': 2515909566090264924}}, 'parent_type': 'global-system-config', 'parent_uuid': 'fe3aad61-f1d2-4bf1-9786-0643d55a11b7', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '22ea4e6b-a5d1-415c-8612-5e7db5913ae1', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.193'}, 'oper': 'CREATE', 'request-id': 'req-79149e01-a0ed-4756-b9e7-1fe46d694b97', 'type': 'virtual_router', 'uuid': '22ea4e6b-a5d1-415c-8612-5e7db5913ae1'} 11/10/2025 05:06:57.309 7f2f8fe96e10 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4678-2'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-4678-2', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4678-2'], 'id_perms': {'created': '2025-11-10T05:06:57.297488', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-10T05:06:57.297488', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11946574162995608735, 'uuid_mslong': 1007975503952693193}}, 'parent_type': 'global-system-config', 'parent_uuid': 'fe3aad61-f1d2-4bf1-9786-0643d55a11b7', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '0dfd0c61-5f65-47c9-a5ca-c1e7d4367c9f', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.129'}, 'oper': 'CREATE', 'request-id': 'req-b8c2dbe4-7d6a-44ed-8566-6c6b91f2b543', 'type': 'virtual_router', 'uuid': '0dfd0c61-5f65-47c9-a5ca-c1e7d4367c9f'} 11/10/2025 05:06:57.643 7f2f8fe96e10 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4678-1', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4678-1', 'vhost0'], 'id_perms': {'created': '2025-11-10T05:06:57.618503', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-10T05:06:57.618503', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13648626880360416446, 'uuid_mslong': 5573548025544196798}}, 'parent_type': 'virtual-router', 'parent_uuid': '22ea4e6b-a5d1-415c-8612-5e7db5913ae1', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '4d593866-3924-42be-bd69-a9d58bbb04be', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:4d:59:38:66:39']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': '5ca3f56e-afb5-4fe1-99a1-670ef1a15aa0'}]}, 'oper': 'CREATE', 'request-id': 'req-2f3feb1b-6bff-48b2-b559-403b7f00468e', 'type': 'virtual_machine_interface', 'uuid': '4d593866-3924-42be-bd69-a9d58bbb04be'} 11/10/2025 05:06:57.708 7f2f8fe96e10 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4678-1', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-2f3feb1b-6bff-48b2-b559-403b7f00468e', 'type': 'virtual_machine_interface', 'uuid': '4d593866-3924-42be-bd69-a9d58bbb04be'} 11/10/2025 05:06:57.782 7f2f8fe96e10 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4678-2', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4678-2', 'vhost0'], 'id_perms': {'created': '2025-11-10T05:06:57.736854', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-10T05:06:57.736854', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11028460101240272815, 'uuid_mslong': 4015097042422156483}}, 'parent_type': 'virtual-router', 'parent_uuid': '0dfd0c61-5f65-47c9-a5ca-c1e7d4367c9f', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '37b87d7c-b84b-4cc3-990c-f607e1feb7af', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:37:b8:7d:7c:b8']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': '5ca3f56e-afb5-4fe1-99a1-670ef1a15aa0'}]}, 'oper': 'CREATE', 'request-id': 'req-5727e3a6-f84b-4c9b-b2e8-e97a85b621ac', 'type': 'virtual_machine_interface', 'uuid': '37b87d7c-b84b-4cc3-990c-f607e1feb7af'} 11/10/2025 05:06:57.837 7f2f8fe96e10 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4678-2', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-5727e3a6-f84b-4c9b-b2e8-e97a85b621ac', 'type': 'virtual_machine_interface', 'uuid': '37b87d7c-b84b-4cc3-990c-f607e1feb7af'} 11/10/2025 05:06:58.218 7f2f8fe96e10 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-74ebf552-f087-4b02-bbe3-9247b1431df9', 'type': 'access_control_list', 'uuid': 'f4accfd9-57ac-441e-8fb8-2490047e5189'} 11/10/2025 05:07:29.251 7f2f8ffb9cc0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 11/10/2025 05:07:58.891 7f2f8ffb9cc0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 11/10/2025 05:09:20.882 7f2f8fe99588 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map