06/23/2025 00:41:33.021 7f54aaf30548 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 06/23/2025 00:41:33.021 7f54aaf30548 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 06/23/2025 00:41:33.021 7f54aaf30548 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 06/23/2025 00:41:33.022 7f54a261b2e8 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 06/23/2025 00:41:33.022 7f54a261b2e8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 06/23/2025 00:41:33.023 7f54a261b2e8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 06/23/2025 00:41:33.023 7f54a261b2e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 06/23/2025 00:41:33.023 7f54a261b2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:33.023 7f54a261b2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 06/23/2025 00:41:33.024 7f54a261b2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:33.024 7f54a261b2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:33.024 7f54a261b2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1750639293023177 collector_name = collector_ip = collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 06/23/2025 00:41:33.024 7f54a261b2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:33.070 7f54aaf30548 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.49:5673//', 'pyamqp://guest:guest@10.0.0.19:5673//', 'pyamqp://guest:guest@10.0.0.41:5673//'] 06/23/2025 00:41:33.079 7f54aaf30548 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 06/23/2025 00:41:33.080 7f54a2574588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:33.080 7f54a2574588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 06/23/2025 00:41:33.080 7f54a2574588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:33.080 7f54a2574588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:33.080 7f54a2574588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 06/23/2025 00:41:33.080 7f54a2574588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:33.086 7f54aaf30548 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 06/23/2025 00:41:33.086 7f54a2574240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:33.087 7f54a2574240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 06/23/2025 00:41:33.087 7f54a2574240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:33.106 7f54aaf30548 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 06/23/2025 00:41:33.107 7f54a25740f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:33.124 7f54a25740f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 06/23/2025 00:41:33.126 7f54a25740f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:33.131 7f54aaf30548 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 06/23/2025 00:41:33.133 7f54b0e20978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:33.133 7f54b0e20978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 06/23/2025 00:41:33.133 7f54b0e20978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:33.143 7f54aaf30548 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 06/23/2025 00:41:33.144 7f54aaf30548 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 06/23/2025 00:41:33.144 7f54aaf30548 [contrail-device-manager] [WARNING]: initializing cassandra cluster 06/23/2025 00:41:33.144 7f54aaf30548 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 06/23/2025 00:41:33.145 7f54a7ea66d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 06/23/2025 00:41:33.145 7f54a7ea66d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 06/23/2025 00:41:33.145 7f54a4db5978 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 06/23/2025 00:41:33.147 7f54a4db5978 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 06/23/2025 00:41:33.148 7f54a2409588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:33.148 7f54a2409588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 06/23/2025 00:41:33.148 7f54a2409588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:33.149 7f54aaf30548 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 06/23/2025 00:41:33.154 7f54a4db5978 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 06/23/2025 00:41:33.157 7f54a7ea66d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 06/23/2025 00:41:33.157 7f54a7ea66d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 06/23/2025 00:41:33.199 7f54aaf30548 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 06/23/2025 00:41:33.202 7f54a23a6cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:33.203 7f54a23a6cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 06/23/2025 00:41:33.203 7f54a23a6cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:33.214 7f54aaf30548 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 06/23/2025 00:41:33.214 7f54a239c2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:33.215 7f54a239c2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 06/23/2025 00:41:33.215 7f54a239c2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:33.223 7f54aaf30548 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 06/23/2025 00:41:33.258 7f54aaf30548 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 06/23/2025 00:41:35.224 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/23/2025 00:41:36.189 7f54aaf30548 [contrail-device-manager] [WARNING]: table 'obj_uuid_table', created 06/23/2025 00:41:36.196 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/23/2025 00:41:37.023 7f54a22d2f60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:41:37.024 7f54a22d2f60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:41:37.024 7f54a22d2f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:41:37.024 7f54a22d2f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:37.025 7f54a22d2f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 06/23/2025 00:41:37.025 7f54a22d2f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:37.025 7f54a22d2f60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:41:37.025 7f54a22d2f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:41:37.025 7f54a22d2f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:37.025 7f54a22d2f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 06/23/2025 00:41:37.025 7f54a22d2f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:37.025 7f54a22d2f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:37.025 7f54a22d2f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:41:37.025 7f54a22d2f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:38.255 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/23/2025 00:41:39.175 7f54aaf30548 [contrail-device-manager] [WARNING]: table 'obj_uuid_table' fixed with properties {'gc_grace_seconds': 864000} 06/23/2025 00:41:39.179 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/23/2025 00:41:41.025 7f54a22e5630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:41:41.027 7f54a22e5630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:41:41.027 7f54a22e5630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:41:41.027 7f54a22e5630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:41.027 7f54a22e5630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 06/23/2025 00:41:41.027 7f54a22e5630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:41.028 7f54a22e5630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:41:41.028 7f54a22e5630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:41:41.028 7f54a22e5630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:41.028 7f54a22e5630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 06/23/2025 00:41:41.028 7f54a22e5630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:41.028 7f54a22e5630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:41.028 7f54a22e5630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:41:41.028 7f54a22e5630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:41.190 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/23/2025 00:41:42.167 7f54aaf30548 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table', created 06/23/2025 00:41:42.172 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/23/2025 00:41:43.343 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/23/2025 00:41:44.050 7f54aaf30548 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table' fixed with properties {'gc_grace_seconds': 864000} 06/23/2025 00:41:44.055 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/23/2025 00:41:45.028 7f54a2391588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:41:45.030 7f54a2391588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:41:45.030 7f54a2391588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:41:45.030 7f54a2391588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:45.030 7f54a2391588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 06/23/2025 00:41:45.030 7f54a2391588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:45.030 7f54a2391588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:41:45.030 7f54a2391588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:41:45.031 7f54a2391588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:45.031 7f54a2391588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 06/23/2025 00:41:45.031 7f54a2391588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:45.031 7f54a2391588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:45.031 7f54a2391588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:41:45.031 7f54a2391588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:46.304 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/23/2025 00:41:47.246 7f54aaf30548 [contrail-device-manager] [WARNING]: table 'obj_shared_table', created 06/23/2025 00:41:47.251 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/23/2025 00:41:48.068 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/23/2025 00:41:49.031 7f54a3153438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:41:49.032 7f54a3153438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:41:49.032 7f54a3153438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:41:49.032 7f54a3153438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:49.033 7f54a3153438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 06/23/2025 00:41:49.033 7f54a3153438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:49.033 7f54a3153438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:41:49.034 7f54a3153438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:41:49.034 7f54a3153438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:49.034 7f54a3153438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 06/23/2025 00:41:49.034 7f54a3153438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:49.034 7f54a3153438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:49.034 7f54a3153438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:41:49.034 7f54a3153438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:49.190 7f54aaf30548 [contrail-device-manager] [WARNING]: table 'obj_shared_table' fixed with properties {'gc_grace_seconds': 864000} 06/23/2025 00:41:49.195 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/23/2025 00:41:49.195 7f54a31b84e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:49.195 7f54a31b84e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:41:49.196 7f54a31b84e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:49.214 7f54aaf30548 [contrail-device-manager] [ERROR]: Error in db walk read 'type' 06/23/2025 00:41:49.216 7f54a22e5a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:49.216 7f54a22e5a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Zookeeper:Zookeeper[] connection down >>, ] >> 06/23/2025 00:41:49.216 7f54a22e5a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:49.224 7f54a31cccc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:49.225 7f54a31cccc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:41:49.225 7f54a31cccc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:49.226 7f54aaf30548 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 06/23/2025 00:41:49.226 7f54aaf30548 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 06/23/2025 00:41:49.226 7f54a7ea66d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 06/23/2025 00:41:49.227 7f54aaf30548 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3515-3. 06/23/2025 00:41:49.238 7f54a7ea66d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 06/23/2025 00:41:49.507 7f54a7ea66d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3515-3.'] 06/23/2025 00:41:49.524 7f54a7ea66d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 06/23/2025 00:41:49.539 7f54a7ea66d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 06/23/2025 00:41:49.601 7f54a7ea66d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3515-3.'] 06/23/2025 00:41:49.739 7f54aaf30548 [contrail-device-manager] [DEBUG]: Fetching all containers 06/23/2025 00:41:50.332 7f54aaf30548 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 06/23/2025 00:41:50.536 7f54aaf30548 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3515-3..config_queue 06/23/2025 00:41:50.536 7f54aaf30548 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3515-3..tftp_queue 06/23/2025 00:41:50.629 7f54a7ea66d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 06/23/2025 00:41:50.648 7f54a7ea66d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 06/23/2025 00:41:50.760 7f54a7ea66d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3515-3.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3515-3..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3515-3..tftp_queue'] 06/23/2025 00:41:51.037 7f54aaf30548 [contrail-device-manager] [WARNING]: Process 140 prepared to run in Full mode... 06/23/2025 00:41:51.038 7f54aaf30548 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 06/23/2025 00:41:51.038 7f54aaf30548 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 06/23/2025 00:41:51.041 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page 06/23/2025 00:41:51.041 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 06/23/2025 00:41:51.041 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page 06/23/2025 00:41:51.042 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 06/23/2025 00:41:51.042 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 06/23/2025 00:41:51.042 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 06/23/2025 00:41:51.046 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page 06/23/2025 00:41:51.047 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 06/23/2025 00:41:51.047 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 06/23/2025 00:41:51.047 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 06/23/2025 00:41:51.047 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 06/23/2025 00:41:51.047 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 06/23/2025 00:41:51.047 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 06/23/2025 00:41:51.047 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 06/23/2025 00:41:51.047 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 06/23/2025 00:41:51.047 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 06/23/2025 00:41:51.047 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 06/23/2025 00:41:51.047 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 06/23/2025 00:41:51.047 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 06/23/2025 00:41:51.049 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 06/23/2025 00:41:51.049 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 06/23/2025 00:41:51.049 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 06/23/2025 00:41:51.049 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 06/23/2025 00:41:51.049 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 06/23/2025 00:41:51.049 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 06/23/2025 00:41:51.050 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 06/23/2025 00:41:51.050 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page 06/23/2025 00:41:51.050 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 06/23/2025 00:41:51.050 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 06/23/2025 00:41:51.050 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 06/23/2025 00:41:51.050 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 06/23/2025 00:41:51.050 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 06/23/2025 00:41:51.051 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page 06/23/2025 00:41:51.051 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 06/23/2025 00:41:51.051 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 06/23/2025 00:41:51.051 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 06/23/2025 00:41:51.051 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 06/23/2025 00:41:51.051 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 06/23/2025 00:41:51.052 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 06/23/2025 00:41:51.052 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 06/23/2025 00:41:51.052 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 06/23/2025 00:41:51.052 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 06/23/2025 00:41:51.052 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 06/23/2025 00:41:51.052 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 06/23/2025 00:41:51.052 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 06/23/2025 00:41:51.052 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 06/23/2025 00:41:51.052 7f54aaf30548 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 06/23/2025 00:41:51.055 7f54aaf30548 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 06/23/2025 00:41:51.068 7f54a1a2d240 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 06/23/2025 00:41:51.068 7f54a1a2d240 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 06/23/2025 00:41:51.135 7f54a1208eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:51.135 7f54a1208eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 06/23/2025 00:41:51.135 7f54a1208eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:51.160 7f54a10f1390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:51.161 7f54a10f1390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 06/23/2025 00:41:51.161 7f54a10f1390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:51.797 7f54a7ea66d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 06/23/2025 00:41:51.802 7f54a7ea66d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 06/23/2025 00:41:51.873 7f54a7ea66d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3515-3.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3515-3..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3515-3..tftp_queue', 'device_manager_ztp.ztp_queue'] 06/23/2025 00:41:53.033 7f54a10fc0f0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:41:53.034 7f54a10fc0f0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:41:53.034 7f54a10fc0f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:41:53.034 7f54a10fc0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:53.034 7f54a10fc0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 06/23/2025 00:41:53.034 7f54a10fc0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:53.034 7f54a10fc0f0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:41:53.035 7f54a10fc0f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:41:53.035 7f54a10fc0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:53.035 7f54a10fc0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 06/23/2025 00:41:53.035 7f54a10fc0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:53.035 7f54a10fc0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:53.035 7f54a10fc0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:41:53.035 7f54a10fc0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:57.034 7f54a102c438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:41:57.036 7f54a102c438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:41:57.036 7f54a102c438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:41:57.036 7f54a102c438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:57.036 7f54a102c438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 06/23/2025 00:41:57.036 7f54a102c438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:57.036 7f54a102c438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:41:57.036 7f54a102c438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:41:57.037 7f54a102c438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:57.037 7f54a102c438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 06/23/2025 00:41:57.037 7f54a102c438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:57.037 7f54a102c438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:57.037 7f54a102c438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:41:57.037 7f54a102c438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:00.323 7f54a0fc24e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:00.323 7f54a0fc24e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:00.324 7f54a0fc24e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:01.037 7f54a0f3c780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:01.038 7f54a0f3c780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:01.038 7f54a0f3c780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:01.038 7f54a0f3c780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:01.039 7f54a0f3c780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:01.039 7f54a0f3c780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:01.039 7f54a0f3c780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:01.039 7f54a0f3c780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:01.039 7f54a0f3c780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:01.039 7f54a0f3c780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:01.039 7f54a0f3c780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:01.039 7f54a0f3c780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:01.039 7f54a0f3c780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:01.039 7f54a0f3c780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:05.038 7f54a0f4ceb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:05.040 7f54a0f4ceb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:05.040 7f54a0f4ceb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:05.040 7f54a0f4ceb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:05.040 7f54a0f4ceb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:05.040 7f54a0f4ceb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:05.041 7f54a0f4ceb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:05.041 7f54a0f4ceb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:05.041 7f54a0f4ceb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:05.041 7f54a0f4ceb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:05.041 7f54a0f4ceb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:05.041 7f54a0f4ceb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:05.041 7f54a0f4ceb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:05.042 7f54a0f4ceb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:09.041 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:09.042 7f54a243f390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:09.042 7f54a243f390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:09.042 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:09.043 7f54a243f390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:09.043 7f54a243f390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:09.043 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:09.043 7f54a243f390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:09.043 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:09.043 7f54a243f390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:09.043 7f54a243f390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:09.043 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:09.044 7f54a243f390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:09.044 7f54a243f390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:13.043 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:13.045 7f54a243f390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:13.045 7f54a243f390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:13.045 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:13.045 7f54a243f390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:13.045 7f54a243f390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:13.045 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:13.045 7f54a243f390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:13.046 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:13.046 7f54a243f390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:13.046 7f54a243f390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:13.046 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:13.046 7f54a243f390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:13.046 7f54a243f390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:17.046 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:17.047 7f54a243f390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:17.047 7f54a243f390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:17.047 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:17.048 7f54a243f390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:17.048 7f54a243f390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:17.048 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:17.048 7f54a243f390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:17.048 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:17.048 7f54a243f390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:17.048 7f54a243f390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:17.048 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:17.049 7f54a243f390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:17.049 7f54a243f390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:21.048 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:21.049 7f54a243f390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:21.050 7f54a243f390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:21.050 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:21.050 7f54a243f390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:21.050 7f54a243f390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:21.050 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:21.050 7f54a243f390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:21.050 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:21.050 7f54a243f390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:21.050 7f54a243f390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:21.050 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:21.051 7f54a243f390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:21.051 7f54a243f390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:25.050 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:25.051 7f54a243f390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:25.052 7f54a243f390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:25.052 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:25.052 7f54a243f390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:25.052 7f54a243f390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:25.052 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:25.052 7f54a243f390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:25.052 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:25.052 7f54a243f390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:25.052 7f54a243f390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:25.053 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:25.053 7f54a243f390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:25.053 7f54a243f390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:29.052 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:29.054 7f54a243f390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:29.054 7f54a243f390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:29.054 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:29.054 7f54a243f390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:29.054 7f54a243f390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:29.054 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:29.054 7f54a243f390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:29.054 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:29.055 7f54a243f390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:29.055 7f54a243f390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:29.055 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:29.055 7f54a243f390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:29.055 7f54a243f390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:33.054 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:33.056 7f54a243f390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:33.056 7f54a243f390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:33.056 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:33.056 7f54a243f390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:33.056 7f54a243f390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:33.056 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:33.056 7f54a243f390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:33.056 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:33.057 7f54a243f390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:33.057 7f54a243f390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:33.057 7f54a243f390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:33.057 7f54a243f390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:33.057 7f54a243f390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:37.057 7f54a22f5978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:37.058 7f54a22f5978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:37.058 7f54a22f5978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:37.058 7f54a22f5978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:37.058 7f54a22f5978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:37.058 7f54a22f5978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:37.059 7f54a22f5978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:37.059 7f54a22f5978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:37.059 7f54a22f5978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:37.059 7f54a22f5978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:37.059 7f54a22f5978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:37.059 7f54a22f5978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:37.059 7f54a22f5978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:37.059 7f54a22f5978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:41.059 7f54a22f5978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:41.060 7f54a22f5978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:41.060 7f54a22f5978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:41.060 7f54a22f5978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:41.060 7f54a22f5978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:41.060 7f54a22f5978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:41.060 7f54a22f5978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:41.061 7f54a22f5978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:41.061 7f54a22f5978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:41.061 7f54a22f5978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:41.061 7f54a22f5978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:41.061 7f54a22f5978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:41.061 7f54a22f5978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:41.061 7f54a22f5978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:45.061 7f54a1078828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:45.062 7f54a1078828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:45.063 7f54a1078828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:45.063 7f54a1078828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:45.063 7f54a1078828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:45.063 7f54a1078828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:45.063 7f54a1078828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:45.064 7f54a1078828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:45.064 7f54a1078828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:45.064 7f54a1078828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:45.064 7f54a1078828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:45.064 7f54a1078828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:45.064 7f54a1078828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:45.064 7f54a1078828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:49.064 7f54a806d438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:49.065 7f54a806d438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:49.065 7f54a806d438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:49.065 7f54a806d438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:49.065 7f54a806d438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:49.065 7f54a806d438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:49.066 7f54a806d438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:49.066 7f54a806d438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:49.066 7f54a806d438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:49.066 7f54a806d438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:49.066 7f54a806d438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:49.066 7f54a806d438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:49.066 7f54a806d438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:49.066 7f54a806d438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:53.066 7f54a19c8198 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:53.067 7f54a19c8198 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:53.067 7f54a19c8198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:53.067 7f54a19c8198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:53.067 7f54a19c8198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:53.067 7f54a19c8198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:53.068 7f54a19c8198 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:53.068 7f54a19c8198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:53.068 7f54a19c8198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:53.068 7f54a19c8198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:53.068 7f54a19c8198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:53.068 7f54a19c8198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:53.068 7f54a19c8198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:53.069 7f54a19c8198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:57.067 7f54a19c8390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:57.068 7f54a19c8390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:57.068 7f54a19c8390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:57.069 7f54a19c8390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:57.069 7f54a19c8390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:57.069 7f54a19c8390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:57.069 7f54a19c8390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:57.069 7f54a19c8390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:57.069 7f54a19c8390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:57.069 7f54a19c8390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:57.069 7f54a19c8390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:57.069 7f54a19c8390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:57.070 7f54a19c8390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:57.070 7f54a19c8390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:01.069 7f54a19c8198 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:01.071 7f54a19c8198 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:43:01.071 7f54a19c8198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:01.071 7f54a19c8198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:01.072 7f54a19c8198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:43:01.072 7f54a19c8198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:01.072 7f54a19c8198 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:43:01.072 7f54a19c8198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:43:01.072 7f54a19c8198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:01.072 7f54a19c8198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:01.072 7f54a19c8198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:01.072 7f54a19c8198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:01.072 7f54a19c8198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:43:01.072 7f54a19c8198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:05.071 7f54a0f3b438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:05.073 7f54a0f3b438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:43:05.073 7f54a0f3b438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:05.073 7f54a0f3b438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:05.073 7f54a0f3b438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:43:05.073 7f54a0f3b438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:05.073 7f54a0f3b438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:43:05.074 7f54a0f3b438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:43:05.074 7f54a0f3b438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:05.074 7f54a0f3b438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:05.074 7f54a0f3b438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:05.074 7f54a0f3b438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:05.074 7f54a0f3b438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:43:05.074 7f54a0f3b438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:09.074 7f54a0f4c6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:09.075 7f54a0f4c6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:43:09.075 7f54a0f4c6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:09.075 7f54a0f4c6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:09.075 7f54a0f4c6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:43:09.075 7f54a0f4c6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:09.075 7f54a0f4c6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:43:09.076 7f54a0f4c6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:43:09.076 7f54a0f4c6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:09.076 7f54a0f4c6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:09.076 7f54a0f4c6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:09.076 7f54a0f4c6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:09.076 7f54a0f4c6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:43:09.076 7f54a0f4c6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:13.076 7f54a80f1828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:13.077 7f54a80f1828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:43:13.077 7f54a80f1828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:13.077 7f54a80f1828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:13.078 7f54a80f1828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:43:13.078 7f54a80f1828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:13.078 7f54a80f1828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:43:13.078 7f54a80f1828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:43:13.078 7f54a80f1828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:13.078 7f54a80f1828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:13.078 7f54a80f1828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:13.078 7f54a80f1828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:13.078 7f54a80f1828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:43:13.078 7f54a80f1828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:17.078 7f54a3153438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:17.079 7f54a3153438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:43:17.079 7f54a3153438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:17.080 7f54a3153438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:17.080 7f54a3153438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:43:17.080 7f54a3153438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:17.080 7f54a3153438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:43:17.081 7f54a3153438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:43:17.081 7f54a3153438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:17.081 7f54a3153438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:17.081 7f54a3153438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:17.081 7f54a3153438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:17.081 7f54a3153438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:43:17.081 7f54a3153438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:18.428 7f54aaf30548 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 06/23/2025 00:43:18.428 7f54aaf30548 [contrail-device-manager] [WARNING]: initializing cassandra cluster 06/23/2025 00:43:18.429 7f54aaf30548 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 06/23/2025 00:43:18.429 7f54a92abd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:18.429 7f54a92abd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 06/23/2025 00:43:18.429 7f54a92abd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:18.430 7f54aaf30548 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 06/23/2025 00:43:18.461 7f54aaf30548 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 06/23/2025 00:43:18.463 7f54a11578d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:18.464 7f54a11578d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Initializing description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 06/23/2025 00:43:18.464 7f54a11578d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:18.471 7f54aaf30548 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 06/23/2025 00:43:18.471 7f54a11576d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:18.471 7f54a11576d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 06/23/2025 00:43:18.471 7f54a11576d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:18.483 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/23/2025 00:43:19.177 7f54aaf30548 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace', created 06/23/2025 00:43:19.196 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/23/2025 00:43:19.222 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/23/2025 00:43:20.154 7f54aaf30548 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None} 06/23/2025 00:43:20.160 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/23/2025 00:43:20.160 7f54aaf30548 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 06/23/2025 00:43:20.307 7f54aaf30548 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 06/23/2025 00:43:20.319 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/23/2025 00:43:21.080 7f54a1140cc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:21.081 7f54a1140cc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:43:21.081 7f54a1140cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:21.081 7f54a1140cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:21.081 7f54a1140cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 06/23/2025 00:43:21.081 7f54a1140cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:21.081 7f54a1140cc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:43:21.081 7f54a1140cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:43:21.082 7f54a1140cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:21.082 7f54a1140cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 06/23/2025 00:43:21.082 7f54a1140cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:21.082 7f54a1140cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:21.082 7f54a1140cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:43:21.082 7f54a1140cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:21.140 7f54aaf30548 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table', created 06/23/2025 00:43:21.144 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/23/2025 00:43:21.151 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/23/2025 00:43:21.300 7f54aaf30548 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table' fixed with properties {'gc_grace_seconds': 864000} 06/23/2025 00:43:21.304 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/23/2025 00:43:21.312 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/23/2025 00:43:22.273 7f54aaf30548 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table', created 06/23/2025 00:43:22.277 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/23/2025 00:43:22.293 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/23/2025 00:43:23.200 7f54aaf30548 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table' fixed with properties {'gc_grace_seconds': 864000} 06/23/2025 00:43:23.204 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/23/2025 00:43:23.210 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/23/2025 00:43:24.215 7f54aaf30548 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table', created 06/23/2025 00:43:24.219 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/23/2025 00:43:24.226 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/23/2025 00:43:25.081 7f54a11a5eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:25.086 7f54a11a5eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:43:25.086 7f54a11a5eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:25.086 7f54a11a5eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:25.087 7f54a11a5eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 06/23/2025 00:43:25.087 7f54a11a5eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:25.087 7f54a11a5eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:43:25.087 7f54a11a5eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:43:25.087 7f54a11a5eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:25.087 7f54a11a5eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 06/23/2025 00:43:25.087 7f54a11a5eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:25.087 7f54a11a5eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:25.087 7f54a11a5eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:43:25.088 7f54a11a5eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:25.376 7f54aaf30548 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table' fixed with properties {'gc_grace_seconds': 864000} 06/23/2025 00:43:25.380 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/23/2025 00:43:25.385 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/23/2025 00:43:26.125 7f54aaf30548 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table', created 06/23/2025 00:43:26.130 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/23/2025 00:43:26.137 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/23/2025 00:43:27.167 7f54aaf30548 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table' fixed with properties {'gc_grace_seconds': 864000} 06/23/2025 00:43:27.171 7f54aaf30548 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/23/2025 00:43:27.172 7f54a0f90630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:27.172 7f54a0f90630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:27.172 7f54a0f90630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:27.241 7f54aaf30548 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.49:5673//', 'pyamqp://guest:guest@10.0.0.19:5673//', 'pyamqp://guest:guest@10.0.0.41:5673//'] 06/23/2025 00:43:27.242 7f54aaf30548 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 06/23/2025 00:43:27.242 7f54a0fa58d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:27.243 7f54a0fa58d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Initializing description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 06/23/2025 00:43:27.243 7f54a0fa58d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:27.243 7f54a0fa58d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:27.243 7f54a0fa58d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 06/23/2025 00:43:27.243 7f54a0fa58d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:27.250 7f54aaf30548 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 06/23/2025 00:43:27.251 7f54a0fa5f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:27.251 7f54a0fa5f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:27.251 7f54a0fa5f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:27.366 7f54aaf30548 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 06/23/2025 00:43:27.367 7f54a0fa2438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:27.368 7f54a0fa2438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 06/23/2025 00:43:27.368 7f54a0fa2438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:27.373 7f54aaf30548 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 06/23/2025 00:43:27.373 7f54a0fa2cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:27.374 7f54a0fa2cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:27.374 7f54a0fa2cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:27.376 7f54a0fa2eb8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 06/23/2025 00:43:27.377 7f54a0f92f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:27.377 7f54a0f92f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 06/23/2025 00:43:27.377 7f54a0f92f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:27.390 7f54a0fa2eb8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 06/23/2025 00:43:27.390 7f54a0f6ee10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:27.391 7f54a0f6ee10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:27.391 7f54a0f6ee10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:29.087 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:29.089 7f54a0f99ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:43:29.089 7f54a0f99ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:29.089 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:29.089 7f54a0f99ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:43:29.089 7f54a0f99ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:29.089 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:43:29.090 7f54a0f99ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:43:29.090 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:29.090 7f54a0f99ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:29.090 7f54a0f99ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:29.090 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:29.090 7f54a0f99ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:43:29.090 7f54a0f99ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:33.089 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:33.090 7f54a0f99ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:43:33.091 7f54a0f99ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:33.091 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:33.091 7f54a0f99ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:43:33.091 7f54a0f99ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:33.091 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:43:33.091 7f54a0f99ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:43:33.091 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:33.092 7f54a0f99ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:33.092 7f54a0f99ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:33.092 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:33.092 7f54a0f99ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:43:33.092 7f54a0f99ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:37.091 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:37.092 7f54a0f99ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:43:37.093 7f54a0f99ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:37.093 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:37.093 7f54a0f99ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:43:37.093 7f54a0f99ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:37.093 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:43:37.093 7f54a0f99ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:43:37.093 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:37.093 7f54a0f99ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:37.093 7f54a0f99ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:37.094 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:37.094 7f54a0f99ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:43:37.094 7f54a0f99ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:41.093 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:41.094 7f54a0f99ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:43:41.095 7f54a0f99ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:41.095 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:41.095 7f54a0f99ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:43:41.095 7f54a0f99ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:41.095 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:43:41.095 7f54a0f99ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:43:41.095 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:41.095 7f54a0f99ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:41.096 7f54a0f99ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:41.096 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:41.096 7f54a0f99ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:43:41.096 7f54a0f99ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:45.095 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:45.096 7f54a0f99ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:43:45.096 7f54a0f99ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:45.097 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:45.097 7f54a0f99ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:43:45.097 7f54a0f99ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:45.097 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:43:45.097 7f54a0f99ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:43:45.097 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:45.098 7f54a0f99ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:45.098 7f54a0f99ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:45.098 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:45.098 7f54a0f99ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:43:45.098 7f54a0f99ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:49.097 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:49.099 7f54a0f99ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:43:49.099 7f54a0f99ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:49.100 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:49.100 7f54a0f99ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:43:49.100 7f54a0f99ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:49.100 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:43:49.100 7f54a0f99ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:43:49.100 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:49.100 7f54a0f99ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:49.100 7f54a0f99ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:49.101 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:49.101 7f54a0f99ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1750639293023177 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.19:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:43:49.101 7f54a0f99ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:53.100 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:53.101 7f54a0f99ac8 [contrail-device-manager] [INFO]: Session Event: TCP Connected 06/23/2025 00:43:53.101 7f54a0f99ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:53.101 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:53.102 7f54a0f99ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.49:5673, 10.0.0.19:5673, 10.0.0.41:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.49:9041, 10.0.0.19:9041, 10.0.0.41:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.49:2181, 10.0.0.19:2181, 10.0.0.41:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.49,10.0.0.19,10.0.0.41:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:43:53.102 7f54a0f99ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:53.102 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect] 06/23/2025 00:43:53.102 7f54a0f99ac8 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19 06/23/2025 00:43:53.102 7f54a0f99ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 06/23/2025 00:43:53.102 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 06/23/2025 00:43:53.102 7f54a0f99ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 06/23/2025 00:43:53.102 7f54a0f9bb70 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 06/23/2025 00:43:53.103 7f54a0f9bb70 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 06/23/2025 00:43:54.028 7f54a0fa2cc0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3515-3.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.41', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3515-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3515-3.'], 'id_perms': {'created': '2025-06-23T00:43:54.010475', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-06-23T00:43:54.010475', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12708961969337214643, 'uuid_mslong': 4026000194446183958}}, 'parent_type': 'global-system-config', 'parent_uuid': '6bb1ba73-f251-45cc-b691-850b471029f6', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '37df39d8-9f9e-4e16-b05f-4d937e4b26b3'}, 'oper': 'CREATE', 'request-id': 'req-066d019e-a5e6-47d3-a4dd-a550eaa8f4a3', 'type': 'analytics_node', 'uuid': '37df39d8-9f9e-4e16-b05f-4d937e4b26b3'} 06/23/2025 00:43:54.089 7f54a0fa2cc0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3515-1.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.49', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3515-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3515-1.'], 'id_perms': {'created': '2025-06-23T00:43:54.070453', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-06-23T00:43:54.070453', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13109025468951803924, 'uuid_mslong': 2169944918933785642}}, 'parent_type': 'global-system-config', 'parent_uuid': '6bb1ba73-f251-45cc-b691-850b471029f6', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '1e1d3168-2b36-4c2a-b5ec-9d359c6bb814'}, 'oper': 'CREATE', 'request-id': 'req-3feffae9-f130-49e5-9ab9-fb014b105312', 'type': 'analytics_node', 'uuid': '1e1d3168-2b36-4c2a-b5ec-9d359c6bb814'} 06/23/2025 00:43:54.128 7f54a0fa2cc0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3515-2.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.19', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3515-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3515-2.'], 'id_perms': {'created': '2025-06-23T00:43:54.113319', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-06-23T00:43:54.113319', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13147732525442036034, 'uuid_mslong': 9001566972915696345}}, 'parent_type': 'global-system-config', 'parent_uuid': '6bb1ba73-f251-45cc-b691-850b471029f6', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '7cebfd78-3287-42d9-b676-21127cd27142'}, 'oper': 'CREATE', 'request-id': 'req-b08e2a36-a34e-4ca8-beef-bdac6445da00', 'type': 'analytics_node', 'uuid': '7cebfd78-3287-42d9-b676-21127cd27142'} 06/23/2025 00:44:05.655 7f54a0f9b2e8 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001016 06/23/2025 00:44:05.655 7f54a0f9b2e8 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 06/23/2025 00:44:05.656 7f54a2312b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 06/23/2025 00:44:05.656 7f54a2312b70 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 0 06/23/2025 00:44:05.656 7f54a2312b70 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/23/2025 00:44:05.656 7f54a2312b70 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 06/23/2025 00:44:05.656 7f54a2312b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 06/23/2025 00:44:05.657 7f54a0fa1630 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 06/23/2025 00:44:05.658 7f54a0fa1630 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 06/23/2025 00:44:10.973 7f54a0fa2cc0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3515-3.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.41', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3515-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3515-3.'], 'id_perms': {'created': '2025-06-23T00:44:10.957310', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-06-23T00:44:10.957310', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12160487477826040596, 'uuid_mslong': 17892740424311590161}}, 'parent_type': 'global-system-config', 'parent_uuid': '6bb1ba73-f251-45cc-b691-850b471029f6', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'f84fc89d-b95d-4d11-a8c2-baee9b683314'}, 'oper': 'CREATE', 'request-id': 'req-1812e4fe-a2d7-4f93-bb0c-6b3f99e0d7c8', 'type': 'analytics_alarm_node', 'uuid': 'f84fc89d-b95d-4d11-a8c2-baee9b683314'} 06/23/2025 00:44:11.225 7f54a0fa2cc0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3515-2.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.19', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3515-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3515-2.'], 'id_perms': {'created': '2025-06-23T00:44:11.209046', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-06-23T00:44:11.209046', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9254569843692492590, 'uuid_mslong': 16813802843580288545}}, 'parent_type': 'global-system-config', 'parent_uuid': '6bb1ba73-f251-45cc-b691-850b471029f6', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'e956a09a-570e-4e21-806e-d63d7255572e'}, 'oper': 'CREATE', 'request-id': 'req-08f7a756-fea3-4ac5-a17b-6feafe8f766a', 'type': 'analytics_alarm_node', 'uuid': 'e956a09a-570e-4e21-806e-d63d7255572e'} 06/23/2025 00:44:11.276 7f54a0fa2cc0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3515-1.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.49', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3515-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3515-1.'], 'id_perms': {'created': '2025-06-23T00:44:11.256238', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-06-23T00:44:11.256238', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9694050882529846720, 'uuid_mslong': 12865828560775299698}}, 'parent_type': 'global-system-config', 'parent_uuid': '6bb1ba73-f251-45cc-b691-850b471029f6', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'b28c9ae8-f216-4272-8688-2fea97c0e5c0'}, 'oper': 'CREATE', 'request-id': 'req-912f0fc2-93b5-49f7-abcd-d2dd996aff38', 'type': 'analytics_alarm_node', 'uuid': 'b28c9ae8-f216-4272-8688-2fea97c0e5c0'} 06/23/2025 00:44:16.090 7f54a0f9b2e8 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001296 06/23/2025 00:44:16.090 7f54a0f9b2e8 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 06/23/2025 00:44:16.091 7f54a804ea20 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 06/23/2025 00:44:16.091 7f54a804ea20 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 2 06/23/2025 00:44:16.091 7f54a804ea20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/23/2025 00:44:16.091 7f54a804ea20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 06/23/2025 00:44:21.144 7f54a0fa2cc0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3515-2.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.19', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3515-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3515-2.'], 'id_perms': {'created': '2025-06-23T00:44:21.124797', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-06-23T00:44:21.124797', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9547846230648360563, 'uuid_mslong': 12747853263748809849}}, 'parent_type': 'global-system-config', 'parent_uuid': '6bb1ba73-f251-45cc-b691-850b471029f6', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'b0e97900-9a8d-4479-8480-c38f63d90e73'}, 'oper': 'CREATE', 'request-id': 'req-d24c77ec-138b-4478-bcf7-4c9340f47028', 'type': 'analytics_snmp_node', 'uuid': 'b0e97900-9a8d-4479-8480-c38f63d90e73'} 06/23/2025 00:44:21.854 7f54a0fa2cc0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3515-3.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.41', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3515-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3515-3.'], 'id_perms': {'created': '2025-06-23T00:44:21.843501', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-06-23T00:44:21.843501', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11563875179742782399, 'uuid_mslong': 13270576893329100046}}, 'parent_type': 'global-system-config', 'parent_uuid': '6bb1ba73-f251-45cc-b691-850b471029f6', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'b82a8f5f-bb63-4d0e-a07b-23353b8663bf'}, 'oper': 'CREATE', 'request-id': 'req-2a4ceab4-90cc-4ddb-b3cb-513e6f572f34', 'type': 'analytics_snmp_node', 'uuid': 'b82a8f5f-bb63-4d0e-a07b-23353b8663bf'} 06/23/2025 00:44:21.967 7f54a0fa2cc0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3515-1.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.49', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3515-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3515-1.'], 'id_perms': {'created': '2025-06-23T00:44:21.954957', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-06-23T00:44:21.954957', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13461838638572033681, 'uuid_mslong': 15181288412564311769}}, 'parent_type': 'global-system-config', 'parent_uuid': '6bb1ba73-f251-45cc-b691-850b471029f6', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'd2aec560-97a6-4ed9-bad2-0eeafbb6d691'}, 'oper': 'CREATE', 'request-id': 'req-776189e7-774d-4d3e-af6c-1316049ef1e8', 'type': 'analytics_snmp_node', 'uuid': 'd2aec560-97a6-4ed9-bad2-0eeafbb6d691'} 06/23/2025 00:46:02.589 7f54a0fa2cc0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-258fbfcc-8b53-4345-9655-81a48f3a8058', 'type': 'global_system_config', 'uuid': '6bb1ba73-f251-45cc-b691-850b471029f6'} 06/23/2025 00:46:02.638 7f54a0fa2cc0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-5d428aeb-915f-4a70-b518-493f1fdd4dfa', 'type': 'global_system_config', 'uuid': '6bb1ba73-f251-45cc-b691-850b471029f6'} 06/23/2025 00:46:02.733 7f54a0fa2cc0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3515-1'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-3515-1', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3515-1'], 'id_perms': {'created': '2025-06-23T00:46:02.725790', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-06-23T00:46:02.725790', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12015729256092671198, 'uuid_mslong': 638969630244227909}}, 'parent_type': 'global-system-config', 'parent_uuid': '6bb1ba73-f251-45cc-b691-850b471029f6', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '08de137e-48f8-4b45-a6c0-7218aaac1cde', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.25'}, 'oper': 'CREATE', 'request-id': 'req-8579f9ab-3237-44c9-9061-64b3917f4193', 'type': 'virtual_router', 'uuid': '08de137e-48f8-4b45-a6c0-7218aaac1cde'} 06/23/2025 00:46:02.786 7f54a0fa2cc0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3515-2'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-3515-2', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3515-2'], 'id_perms': {'created': '2025-06-23T00:46:02.777364', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-06-23T00:46:02.777364', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13131383605618136228, 'uuid_mslong': 582756256945293254}}, 'parent_type': 'global-system-config', 'parent_uuid': '6bb1ba73-f251-45cc-b691-850b471029f6', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '08165dba-65dc-47c6-b63c-0bd0f641d4a4', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.193'}, 'oper': 'CREATE', 'request-id': 'req-58db5320-e8ce-43f9-a7b4-84e539cb31bc', 'type': 'virtual_router', 'uuid': '08165dba-65dc-47c6-b63c-0bd0f641d4a4'} 06/23/2025 00:46:03.126 7f54a0fa2cc0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3515-1', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3515-1', 'vhost0'], 'id_perms': {'created': '2025-06-23T00:46:03.087055', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-06-23T00:46:03.087055', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13808408695167283556, 'uuid_mslong': 11698381355008540990}}, 'parent_type': 'virtual-router', 'parent_uuid': '08de137e-48f8-4b45-a6c0-7218aaac1cde', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'a258ffdc-9856-413e-bfa1-528c54dc7164', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:a2:58:ff:dc:98']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': 'e488264b-a209-43d7-b0b4-eab5a1f6df46'}]}, 'oper': 'CREATE', 'request-id': 'req-9fa3ab85-3bd4-4849-8e30-8c5a702d39c2', 'type': 'virtual_machine_interface', 'uuid': 'a258ffdc-9856-413e-bfa1-528c54dc7164'} 06/23/2025 00:46:03.223 7f54a0fa2cc0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3515-1', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-9fa3ab85-3bd4-4849-8e30-8c5a702d39c2', 'type': 'virtual_machine_interface', 'uuid': 'a258ffdc-9856-413e-bfa1-528c54dc7164'} 06/23/2025 00:46:03.266 7f54a0fa2cc0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3515-2', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3515-2', 'vhost0'], 'id_perms': {'created': '2025-06-23T00:46:03.205665', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-06-23T00:46:03.205665', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9806787067834754872, 'uuid_mslong': 5773842293571209534}}, 'parent_type': 'virtual-router', 'parent_uuid': '08165dba-65dc-47c6-b63c-0bd0f641d4a4', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '5020cef9-9170-493e-8818-b4e12e2f6738', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:50:20:ce:f9:91']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': 'e488264b-a209-43d7-b0b4-eab5a1f6df46'}]}, 'oper': 'CREATE', 'request-id': 'req-13e490ab-cf7e-4769-af88-e2655a7f7de3', 'type': 'virtual_machine_interface', 'uuid': '5020cef9-9170-493e-8818-b4e12e2f6738'} 06/23/2025 00:46:03.363 7f54a0fa2cc0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3515-2', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-13e490ab-cf7e-4769-af88-e2655a7f7de3', 'type': 'virtual_machine_interface', 'uuid': '5020cef9-9170-493e-8818-b4e12e2f6738'} 06/23/2025 00:46:03.674 7f54a0fa2cc0 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-670c93af-7ca5-455c-8d45-8589334771db', 'type': 'access_control_list', 'uuid': 'a088b7b8-d3ce-455e-bf96-4890fb5f9514'} 06/23/2025 00:46:43.037 7f54a11bd588 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/23/2025 00:47:02.233 7f54a11bd588 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/23/2025 01:34:42.512 7f54a22faeb8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map