01/16/2025 00:52:58.882 7f12d7c6b988 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 01/16/2025 00:52:58.882 7f12d7c6b988 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 01/16/2025 00:52:58.882 7f12d7c6b988 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 01/16/2025 00:52:58.882 7f12cf363b70 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 01/16/2025 00:52:58.883 7f12cf363b70 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 00:52:58.883 7f12cf363b70 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 01/16/2025 00:52:58.884 7f12cf363b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 01/16/2025 00:52:58.884 7f12cf363b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:52:58.884 7f12cf363b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 01/16/2025 00:52:58.884 7f12cf363b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:52:58.884 7f12cf363b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:52:58.884 7f12cf363b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1736988778883731 collector_name = collector_ip = collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 01/16/2025 00:52:58.884 7f12cf363b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:52:58.938 7f12d7c6b988 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.43:5673//', 'pyamqp://guest:guest@10.0.0.29:5673//', 'pyamqp://guest:guest@10.0.0.23:5673//'] 01/16/2025 00:52:58.947 7f12d7c6b988 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 01/16/2025 00:52:58.948 7f12cf1b7198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:52:58.948 7f12cf1b7198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 01/16/2025 00:52:58.948 7f12cf1b7198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:52:58.948 7f12cf1b7198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:52:58.948 7f12cf1b7198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 01/16/2025 00:52:58.948 7f12cf1b7198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:52:58.953 7f12d7c6b988 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 01/16/2025 00:52:58.953 7f12cf1cf978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:52:58.953 7f12cf1cf978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 01/16/2025 00:52:58.953 7f12cf1cf978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:52:59.007 7f12d7c6b988 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 01/16/2025 00:52:59.008 7f12cf1b72e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:52:59.008 7f12cf1b72e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 01/16/2025 00:52:59.008 7f12cf1b72e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:52:59.013 7f12d7c6b988 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 01/16/2025 00:52:59.013 7f12cf1b7198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:52:59.013 7f12cf1b7198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 01/16/2025 00:52:59.013 7f12cf1b7198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:52:59.044 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 01/16/2025 00:52:59.044 7f12d7c6b988 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 01/16/2025 00:52:59.044 7f12d7c6b988 [contrail-device-manager] [WARNING]: initializing cassandra cluster 01/16/2025 00:52:59.044 7f12d7c6b988 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 01/16/2025 00:52:59.045 7f12ddaa8630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 01/16/2025 00:52:59.045 7f12ddaa8630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 01/16/2025 00:52:59.045 7f12cf1cf8d0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 01/16/2025 00:52:59.046 7f12cf1cf8d0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 01/16/2025 00:52:59.046 7f12cf130438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:52:59.046 7f12cf130438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 01/16/2025 00:52:59.046 7f12cf130438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:52:59.047 7f12d7c6b988 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 01/16/2025 00:52:59.051 7f12ddaa8630 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 01/16/2025 00:52:59.051 7f12ddaa8630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 01/16/2025 00:52:59.051 7f12cf1cf8d0 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 01/16/2025 00:52:59.085 7f12d7c6b988 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 01/16/2025 00:52:59.087 7f12cf0c3630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:52:59.088 7f12cf0c3630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 01/16/2025 00:52:59.088 7f12cf0c3630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:52:59.097 7f12d7c6b988 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 01/16/2025 00:52:59.097 7f12d7c15048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:52:59.097 7f12d7c15048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 01/16/2025 00:52:59.097 7f12d7c15048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:52:59.107 7f12d7c6b988 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 01/16/2025 00:52:59.135 7f12d7c6b988 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 01/16/2025 00:53:00.146 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 01/16/2025 00:53:01.148 7f12d7c6b988 [contrail-device-manager] [WARNING]: table 'obj_uuid_table', created 01/16/2025 00:53:01.155 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock released. 01/16/2025 00:53:01.856 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 01/16/2025 00:53:02.426 7f12d7c6b988 [contrail-device-manager] [WARNING]: table 'obj_uuid_table' fixed with properties {'gc_grace_seconds': 864000} 01/16/2025 00:53:02.432 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock released. 01/16/2025 00:53:02.883 7f12cf00bac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:53:02.884 7f12cf00bac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:53:02.885 7f12cf00bac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:53:02.885 7f12cf00bac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:02.885 7f12cf00bac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 01/16/2025 00:53:02.885 7f12cf00bac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:02.885 7f12cf00bac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:53:02.885 7f12cf00bac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:53:02.885 7f12cf00bac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:02.885 7f12cf00bac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 01/16/2025 00:53:02.885 7f12cf00bac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:02.885 7f12cf00bac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:02.885 7f12cf00bac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:53:02.886 7f12cf00bac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:04.333 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 01/16/2025 00:53:05.327 7f12d7c6b988 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table', created 01/16/2025 00:53:05.333 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock released. 01/16/2025 00:53:06.586 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 01/16/2025 00:53:06.885 7f12cf03f4e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:53:06.887 7f12cf03f4e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:53:06.887 7f12cf03f4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:53:06.887 7f12cf03f4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:06.887 7f12cf03f4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 01/16/2025 00:53:06.887 7f12cf03f4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:06.887 7f12cf03f4e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:53:06.887 7f12cf03f4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:53:06.887 7f12cf03f4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:06.888 7f12cf03f4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 01/16/2025 00:53:06.888 7f12cf03f4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:06.888 7f12cf03f4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:06.888 7f12cf03f4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.29:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:53:06.888 7f12cf03f4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:08.314 7f12d7c6b988 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table' fixed with properties {'gc_grace_seconds': 864000} 01/16/2025 00:53:08.321 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock released. 01/16/2025 00:53:10.750 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 01/16/2025 00:53:10.765 7f12d7c6b988 [contrail-device-manager] [WARNING]: table 'obj_shared_table', created 01/16/2025 00:53:10.773 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock released. 01/16/2025 00:53:10.888 7f12cf01a978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:53:10.889 7f12cf01a978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:53:10.889 7f12cf01a978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:53:10.889 7f12cf01a978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:10.889 7f12cf01a978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 01/16/2025 00:53:10.890 7f12cf01a978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:10.890 7f12cf01a978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:53:10.890 7f12cf01a978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:53:10.890 7f12cf01a978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:10.890 7f12cf01a978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 01/16/2025 00:53:10.890 7f12cf01a978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:10.890 7f12cf01a978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:10.891 7f12cf01a978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.43:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:53:10.891 7f12cf01a978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:12.236 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 01/16/2025 00:53:12.782 7f12d7c6b988 [contrail-device-manager] [WARNING]: table 'obj_shared_table' fixed with properties {'gc_grace_seconds': 864000} 01/16/2025 00:53:12.793 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock released. 01/16/2025 00:53:12.794 7f12ceff6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:12.794 7f12ceff6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:53:12.794 7f12ceff6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:12.809 7f12d7c6b988 [contrail-device-manager] [ERROR]: Error in db walk read 'type' 01/16/2025 00:53:12.810 7f12cf01fd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:12.811 7f12cf01fd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Zookeeper:Zookeeper[] connection down >>, ] >> 01/16/2025 00:53:12.811 7f12cf01fd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:12.820 7f12cf01f438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:12.821 7f12cf01f438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:53:12.821 7f12cf01f438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:12.821 7f12d7c6b988 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 01/16/2025 00:53:12.821 7f12d7c6b988 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 01/16/2025 00:53:12.821 7f12ddaa8630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 01/16/2025 00:53:12.822 7f12d7c6b988 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2646-3. 01/16/2025 00:53:12.830 7f12ddaa8630 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 01/16/2025 00:53:13.085 7f12ddaa8630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2646-3.'] 01/16/2025 00:53:13.116 7f12ddaa8630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 01/16/2025 00:53:13.122 7f12ddaa8630 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 01/16/2025 00:53:13.186 7f12ddaa8630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2646-3.'] 01/16/2025 00:53:13.332 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Fetching all containers 01/16/2025 00:53:13.387 7f12d7c6b988 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 01/16/2025 00:53:13.647 7f12d7c6b988 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2646-3..config_queue 01/16/2025 00:53:13.647 7f12d7c6b988 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2646-3..tftp_queue 01/16/2025 00:53:14.148 7f12d7c6b988 [contrail-device-manager] [WARNING]: Process 140 prepared to run in Full mode... 01/16/2025 00:53:14.149 7f12d7c6b988 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 01/16/2025 00:53:14.149 7f12d7c6b988 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 01/16/2025 00:53:14.152 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page 01/16/2025 00:53:14.152 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 01/16/2025 00:53:14.153 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page 01/16/2025 00:53:14.153 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 01/16/2025 00:53:14.154 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 01/16/2025 00:53:14.154 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 01/16/2025 00:53:14.158 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page 01/16/2025 00:53:14.158 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 01/16/2025 00:53:14.158 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 01/16/2025 00:53:14.158 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 01/16/2025 00:53:14.159 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 01/16/2025 00:53:14.159 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 01/16/2025 00:53:14.159 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 01/16/2025 00:53:14.159 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 01/16/2025 00:53:14.159 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 01/16/2025 00:53:14.159 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 01/16/2025 00:53:14.159 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 01/16/2025 00:53:14.159 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 01/16/2025 00:53:14.159 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 01/16/2025 00:53:14.161 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 01/16/2025 00:53:14.161 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 01/16/2025 00:53:14.161 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 01/16/2025 00:53:14.161 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 01/16/2025 00:53:14.162 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 01/16/2025 00:53:14.162 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 01/16/2025 00:53:14.162 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 01/16/2025 00:53:14.162 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page 01/16/2025 00:53:14.162 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 01/16/2025 00:53:14.162 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 01/16/2025 00:53:14.162 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 01/16/2025 00:53:14.162 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 01/16/2025 00:53:14.163 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 01/16/2025 00:53:14.164 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page 01/16/2025 00:53:14.164 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 01/16/2025 00:53:14.164 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 01/16/2025 00:53:14.164 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 01/16/2025 00:53:14.164 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 01/16/2025 00:53:14.164 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 01/16/2025 00:53:14.164 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 01/16/2025 00:53:14.164 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 01/16/2025 00:53:14.164 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 01/16/2025 00:53:14.164 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 01/16/2025 00:53:14.164 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 01/16/2025 00:53:14.165 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 01/16/2025 00:53:14.165 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 01/16/2025 00:53:14.165 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 01/16/2025 00:53:14.165 7f12d7c6b988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 01/16/2025 00:53:14.167 7f12d7c6b988 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 01/16/2025 00:53:14.180 7f12ce79ae10 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 01/16/2025 00:53:14.180 7f12ce79ae10 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 01/16/2025 00:53:14.225 7f12cdf24630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:14.226 7f12cdf24630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 01/16/2025 00:53:14.226 7f12cdf24630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:14.240 7f12cde79978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:14.240 7f12cde79978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 01/16/2025 00:53:14.241 7f12cde79978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:14.254 7f12ddaa8630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 01/16/2025 00:53:14.260 7f12ddaa8630 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 01/16/2025 00:53:14.548 7f12ddaa8630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2646-3.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2646-3..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2646-3..tftp_queue', 'device_manager_ztp.ztp_queue'] 01/16/2025 00:53:14.890 7f12ce75ac18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:53:14.891 7f12ce75ac18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:53:14.891 7f12ce75ac18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:53:14.891 7f12ce75ac18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:14.891 7f12ce75ac18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 01/16/2025 00:53:14.891 7f12ce75ac18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:14.892 7f12ce75ac18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:53:14.892 7f12ce75ac18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:53:14.892 7f12ce75ac18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:14.892 7f12ce75ac18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 01/16/2025 00:53:14.892 7f12ce75ac18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:14.892 7f12ce75ac18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:14.892 7f12ce75ac18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:53:14.892 7f12ce75ac18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:18.892 7f12cddac8d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:53:18.894 7f12cddac8d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:53:18.894 7f12cddac8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:53:18.894 7f12cddac8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:18.894 7f12cddac8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 01/16/2025 00:53:18.894 7f12cddac8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:18.894 7f12cddac8d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:53:18.895 7f12cddac8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:53:18.895 7f12cddac8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:18.895 7f12cddac8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 01/16/2025 00:53:18.895 7f12cddac8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:18.895 7f12cddac8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:18.895 7f12cddac8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.29:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:53:18.895 7f12cddac8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:20.385 7f12cde048d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:20.385 7f12cde048d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:53:20.385 7f12cde048d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:22.895 7f12cdcbecc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:53:22.896 7f12cdcbecc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:53:22.896 7f12cdcbecc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:53:22.896 7f12cdcbecc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:22.896 7f12cdcbecc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:53:22.897 7f12cdcbecc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:22.897 7f12cdcbecc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:53:22.897 7f12cdcbecc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:53:22.897 7f12cdcbecc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:22.897 7f12cdcbecc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:53:22.897 7f12cdcbecc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:22.897 7f12cdcbecc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:22.898 7f12cdcbecc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.43:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:53:22.898 7f12cdcbecc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:26.897 7f12cdcc7588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:53:26.898 7f12cdcc7588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:53:26.898 7f12cdcc7588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:53:26.898 7f12cdcc7588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:26.898 7f12cdcc7588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:53:26.898 7f12cdcc7588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:26.898 7f12cdcc7588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:53:26.899 7f12cdcc7588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:53:26.899 7f12cdcc7588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:26.899 7f12cdcc7588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:53:26.899 7f12cdcc7588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:26.899 7f12cdcc7588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:26.899 7f12cdcc7588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:53:26.899 7f12cdcc7588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:30.899 7f12cdd5cb70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:53:30.901 7f12cdd5cb70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:53:30.901 7f12cdd5cb70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:53:30.901 7f12cdd5cb70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:30.901 7f12cdd5cb70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:53:30.901 7f12cdd5cb70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:30.902 7f12cdd5cb70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:53:30.902 7f12cdd5cb70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:53:30.902 7f12cdd5cb70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:30.903 7f12cdd5cb70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:53:30.903 7f12cdd5cb70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:30.903 7f12cdd5cb70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:30.903 7f12cdd5cb70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.29:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:53:30.903 7f12cdd5cb70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:34.901 7f12d4d8ccc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:53:34.903 7f12d4d8ccc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:53:34.903 7f12d4d8ccc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:53:34.904 7f12d4d8ccc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:34.904 7f12d4d8ccc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:53:34.904 7f12d4d8ccc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:34.904 7f12d4d8ccc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:53:34.905 7f12d4d8ccc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:53:34.905 7f12d4d8ccc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:34.905 7f12d4d8ccc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:53:34.905 7f12d4d8ccc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:34.905 7f12d4d8ccc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:34.905 7f12d4d8ccc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.43:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:53:34.905 7f12d4d8ccc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:38.903 7f12cdd64048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:53:38.904 7f12cdd64048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:53:38.905 7f12cdd64048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:53:38.905 7f12cdd64048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:38.905 7f12cdd64048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:53:38.905 7f12cdd64048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:38.905 7f12cdd64048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:53:38.906 7f12cdd64048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:53:38.906 7f12cdd64048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:38.906 7f12cdd64048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:53:38.906 7f12cdd64048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:38.906 7f12cdd64048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:38.906 7f12cdd64048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:53:38.906 7f12cdd64048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:42.905 7f12cdcc1048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:53:42.906 7f12cdcc1048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:53:42.907 7f12cdcc1048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:53:42.907 7f12cdcc1048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:42.907 7f12cdcc1048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:53:42.907 7f12cdcc1048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:42.907 7f12cdcc1048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:53:42.907 7f12cdcc1048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:53:42.907 7f12cdcc1048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:42.907 7f12cdcc1048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:53:42.908 7f12cdcc1048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:42.908 7f12cdcc1048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:42.908 7f12cdcc1048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.29:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:53:42.908 7f12cdcc1048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:46.907 7f12cdd268d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:53:46.908 7f12cdd268d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:53:46.908 7f12cdd268d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:53:46.908 7f12cdd268d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:46.909 7f12cdd268d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:53:46.909 7f12cdd268d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:46.909 7f12cdd268d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:53:46.909 7f12cdd268d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:53:46.909 7f12cdd268d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:46.909 7f12cdd268d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:53:46.910 7f12cdd268d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:46.910 7f12cdd268d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:46.910 7f12cdd268d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.43:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:53:46.910 7f12cdd268d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:50.909 7f12d1ce7cc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:53:50.910 7f12d1ce7cc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:53:50.910 7f12d1ce7cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:53:50.910 7f12d1ce7cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:50.910 7f12d1ce7cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:53:50.910 7f12d1ce7cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:50.910 7f12d1ce7cc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:53:50.910 7f12d1ce7cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:53:50.910 7f12d1ce7cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:50.910 7f12d1ce7cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:53:50.910 7f12d1ce7cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:50.911 7f12d1ce7cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:50.911 7f12d1ce7cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:53:50.911 7f12d1ce7cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:54.910 7f12d1ce7cc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:53:54.912 7f12d1ce7cc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:53:54.912 7f12d1ce7cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:53:54.912 7f12d1ce7cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:54.912 7f12d1ce7cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:53:54.912 7f12d1ce7cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:54.912 7f12d1ce7cc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:53:54.913 7f12d1ce7cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:53:54.913 7f12d1ce7cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:54.913 7f12d1ce7cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:53:54.913 7f12d1ce7cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:54.913 7f12d1ce7cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:54.913 7f12d1ce7cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.29:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:53:54.913 7f12d1ce7cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:58.912 7f12cf0f4828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:53:58.914 7f12cf0f4828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:53:58.914 7f12cf0f4828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:53:58.914 7f12cf0f4828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:58.914 7f12cf0f4828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:53:58.915 7f12cf0f4828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:53:58.915 7f12cf0f4828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:53:58.915 7f12cf0f4828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:53:58.915 7f12cf0f4828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:58.915 7f12cf0f4828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:53:58.915 7f12cf0f4828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:58.915 7f12cf0f4828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:53:58.916 7f12cf0f4828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.43:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:53:58.916 7f12cf0f4828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:02.914 7f12d4d8ccc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:54:02.915 7f12d4d8ccc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:54:02.915 7f12d4d8ccc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:54:02.915 7f12d4d8ccc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:02.916 7f12d4d8ccc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:54:02.916 7f12d4d8ccc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:02.916 7f12d4d8ccc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:54:02.916 7f12d4d8ccc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:54:02.916 7f12d4d8ccc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:02.916 7f12d4d8ccc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:54:02.916 7f12d4d8ccc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:02.916 7f12d4d8ccc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:02.917 7f12d4d8ccc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:54:02.917 7f12d4d8ccc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:06.916 7f12cdd64048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:54:06.920 7f12cdd64048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:54:06.920 7f12cdd64048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:54:06.921 7f12cdd64048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:06.921 7f12cdd64048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:54:06.921 7f12cdd64048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:06.921 7f12cdd64048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:54:06.921 7f12cdd64048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:54:06.921 7f12cdd64048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:06.921 7f12cdd64048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:54:06.921 7f12cdd64048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:06.921 7f12cdd64048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:06.922 7f12cdd64048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.29:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:54:06.922 7f12cdd64048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:09.851 7f12d7c6b988 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 01/16/2025 00:54:09.851 7f12d7c6b988 [contrail-device-manager] [WARNING]: initializing cassandra cluster 01/16/2025 00:54:09.852 7f12d7c6b988 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 01/16/2025 00:54:09.852 7f12d5fd9e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:09.852 7f12d5fd9e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 01/16/2025 00:54:09.852 7f12d5fd9e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:09.853 7f12d7c6b988 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 01/16/2025 00:54:09.889 7f12d7c6b988 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 01/16/2025 00:54:09.891 7f12cdec1978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:09.891 7f12cdec1978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Initializing description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 01/16/2025 00:54:09.891 7f12cdec1978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:09.897 7f12d7c6b988 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 01/16/2025 00:54:09.897 7f12cdec1780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:09.897 7f12cdec1780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 01/16/2025 00:54:09.897 7f12cdec1780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:09.916 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 01/16/2025 00:54:10.620 7f12d7c6b988 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace', created 01/16/2025 00:54:10.624 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock released. 01/16/2025 00:54:10.630 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 01/16/2025 00:54:10.921 7f12cdc93eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:54:10.923 7f12cdc93eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:54:10.923 7f12cdc93eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:54:10.923 7f12cdc93eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:10.923 7f12cdc93eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 01/16/2025 00:54:10.923 7f12cdc93eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:10.923 7f12cdc93eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:54:10.924 7f12cdc93eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:54:10.924 7f12cdc93eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:10.924 7f12cdc93eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 01/16/2025 00:54:10.924 7f12cdc93eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:10.924 7f12cdc93eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:10.924 7f12cdc93eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.43:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:54:10.924 7f12cdc93eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:11.530 7f12d7c6b988 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None} 01/16/2025 00:54:11.535 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock released. 01/16/2025 00:54:11.535 7f12d7c6b988 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 01/16/2025 00:54:11.627 7f12d7c6b988 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 01/16/2025 00:54:11.633 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 01/16/2025 00:54:12.646 7f12d7c6b988 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table', created 01/16/2025 00:54:12.651 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock released. 01/16/2025 00:54:12.658 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 01/16/2025 00:54:13.673 7f12d7c6b988 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table' fixed with properties {'gc_grace_seconds': 864000} 01/16/2025 00:54:13.677 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock released. 01/16/2025 00:54:13.683 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 01/16/2025 00:54:14.488 7f12d7c6b988 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table', created 01/16/2025 00:54:14.493 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock released. 01/16/2025 00:54:14.499 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 01/16/2025 00:54:14.924 7f12cdc7d390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:54:14.925 7f12cdc7d390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:54:14.925 7f12cdc7d390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:54:14.925 7f12cdc7d390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:14.925 7f12cdc7d390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 01/16/2025 00:54:14.925 7f12cdc7d390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:14.926 7f12cdc7d390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:54:14.926 7f12cdc7d390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:54:14.926 7f12cdc7d390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:14.926 7f12cdc7d390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 01/16/2025 00:54:14.926 7f12cdc7d390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:14.926 7f12cdc7d390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:14.926 7f12cdc7d390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:54:14.926 7f12cdc7d390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:15.207 7f12d7c6b988 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table' fixed with properties {'gc_grace_seconds': 864000} 01/16/2025 00:54:15.212 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock released. 01/16/2025 00:54:15.218 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 01/16/2025 00:54:16.166 7f12d7c6b988 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table', created 01/16/2025 00:54:16.171 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock released. 01/16/2025 00:54:16.177 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 01/16/2025 00:54:16.715 7f12d7c6b988 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table' fixed with properties {'gc_grace_seconds': 864000} 01/16/2025 00:54:16.720 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock released. 01/16/2025 00:54:16.727 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 01/16/2025 00:54:17.537 7f12d7c6b988 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table', created 01/16/2025 00:54:17.545 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock released. 01/16/2025 00:54:17.551 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 01/16/2025 00:54:18.240 7f12d7c6b988 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table' fixed with properties {'gc_grace_seconds': 864000} 01/16/2025 00:54:18.254 7f12d7c6b988 [contrail-device-manager] [WARNING]: Zookeeper lock released. 01/16/2025 00:54:18.254 7f12cdc006d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:18.255 7f12cdc006d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:54:18.255 7f12cdc006d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:18.319 7f12d7c6b988 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.43:5673//', 'pyamqp://guest:guest@10.0.0.29:5673//', 'pyamqp://guest:guest@10.0.0.23:5673//'] 01/16/2025 00:54:18.319 7f12d7c6b988 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 01/16/2025 00:54:18.320 7f12cdebf390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:18.320 7f12cdebf390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Initializing description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 01/16/2025 00:54:18.320 7f12cdebf390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:18.320 7f12cdebf390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:18.320 7f12cdebf390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 01/16/2025 00:54:18.320 7f12cdebf390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:18.324 7f12d7c6b988 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 01/16/2025 00:54:18.325 7f12cdc8a0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:18.325 7f12cdc8a0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:54:18.325 7f12cdc8a0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:18.378 7f12d7c6b988 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 01/16/2025 00:54:18.378 7f12cdc97cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:18.379 7f12cdc97cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 01/16/2025 00:54:18.379 7f12cdc97cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:18.382 7f12d7c6b988 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 01/16/2025 00:54:18.383 7f12cdc97390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:18.383 7f12cdc97390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:54:18.383 7f12cdc97390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:18.384 7f12cdcc08d0 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 01/16/2025 00:54:18.385 7f12cdc93eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:18.385 7f12cdc93eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 01/16/2025 00:54:18.385 7f12cdc93eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:18.389 7f12cdcc08d0 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 01/16/2025 00:54:18.390 7f12cdc78d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:18.390 7f12cdc78d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:54:18.390 7f12cdc78d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:18.926 7f12cdcf54e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:54:18.928 7f12cdcf54e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:54:18.928 7f12cdcf54e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:54:18.928 7f12cdcf54e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:18.928 7f12cdcf54e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:54:18.928 7f12cdcf54e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:18.928 7f12cdcf54e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:54:18.928 7f12cdcf54e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:54:18.928 7f12cdcf54e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:18.929 7f12cdcf54e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:54:18.929 7f12cdcf54e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:18.929 7f12cdcf54e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:18.929 7f12cdcf54e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.29:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:54:18.929 7f12cdcf54e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:22.929 7f12cdc73d68 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:54:22.930 7f12cdc73d68 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:54:22.931 7f12cdc73d68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:54:22.931 7f12cdc73d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:22.931 7f12cdc73d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:54:22.931 7f12cdc73d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:22.931 7f12cdc73d68 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:54:22.931 7f12cdc73d68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:54:22.931 7f12cdc73d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:22.931 7f12cdc73d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:54:22.932 7f12cdc73d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:22.932 7f12cdc73d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:22.932 7f12cdc73d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.43:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:54:22.932 7f12cdc73d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:26.931 7f12cdc73d68 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:54:26.933 7f12cdc73d68 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:54:26.933 7f12cdc73d68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:54:26.933 7f12cdc73d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:26.933 7f12cdc73d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:54:26.933 7f12cdc73d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:26.933 7f12cdc73d68 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:54:26.934 7f12cdc73d68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:54:26.934 7f12cdc73d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:26.934 7f12cdc73d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:54:26.934 7f12cdc73d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:26.934 7f12cdc73d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:26.934 7f12cdc73d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:54:26.934 7f12cdc73d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:30.933 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:54:30.935 7f12cf037eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:54:30.935 7f12cf037eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:54:30.935 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:30.935 7f12cf037eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:54:30.935 7f12cf037eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:30.935 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:54:30.936 7f12cf037eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:54:30.936 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:30.936 7f12cf037eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:54:30.936 7f12cf037eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:30.936 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:30.936 7f12cf037eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.29:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:54:30.936 7f12cf037eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:34.936 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:54:34.937 7f12cf037eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:54:34.937 7f12cf037eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:54:34.937 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:34.938 7f12cf037eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:54:34.938 7f12cf037eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:34.938 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:54:34.938 7f12cf037eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:54:34.938 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:34.938 7f12cf037eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:54:34.938 7f12cf037eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:34.938 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:34.938 7f12cf037eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.43:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:54:34.939 7f12cf037eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:38.938 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:54:38.939 7f12cf037eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:54:38.940 7f12cf037eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:54:38.940 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:38.940 7f12cf037eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:54:38.940 7f12cf037eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:38.940 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:54:38.940 7f12cf037eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:54:38.940 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:38.941 7f12cf037eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:54:38.941 7f12cf037eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:38.941 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:38.941 7f12cf037eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:54:38.941 7f12cf037eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:42.940 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:54:42.942 7f12cf037eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:54:42.942 7f12cf037eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:54:42.942 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:42.942 7f12cf037eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:54:42.943 7f12cf037eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:42.943 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:54:42.943 7f12cf037eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:54:42.943 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:42.943 7f12cf037eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:54:42.943 7f12cf037eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:42.943 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:42.944 7f12cf037eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.29:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:54:42.944 7f12cf037eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:46.942 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:54:46.944 7f12cf037eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:54:46.944 7f12cf037eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:54:46.944 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:46.945 7f12cf037eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:54:46.945 7f12cf037eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:46.945 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:54:46.945 7f12cf037eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:54:46.945 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:46.945 7f12cf037eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:54:46.945 7f12cf037eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:46.945 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:46.945 7f12cf037eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.43:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:54:46.946 7f12cf037eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:50.945 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:54:50.947 7f12cf037eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:54:50.947 7f12cf037eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:54:50.947 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:50.947 7f12cf037eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:54:50.947 7f12cf037eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:50.947 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:54:50.947 7f12cf037eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:54:50.948 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:50.948 7f12cf037eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:54:50.948 7f12cf037eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:50.948 7f12cf037eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:50.948 7f12cf037eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:54:50.948 7f12cf037eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:54.947 7f12cf1b74e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:54:54.949 7f12cf1b74e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:54:54.949 7f12cf1b74e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:54:54.949 7f12cf1b74e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:54.949 7f12cf1b74e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:54:54.950 7f12cf1b74e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:54.950 7f12cf1b74e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:54:54.950 7f12cf1b74e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:54:54.950 7f12cf1b74e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:54.950 7f12cf1b74e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:54:54.950 7f12cf1b74e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:54.950 7f12cf1b74e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:54.950 7f12cf1b74e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.29:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:54:54.950 7f12cf1b74e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:58.950 7f12cf1b74e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:54:58.951 7f12cf1b74e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:54:58.952 7f12cf1b74e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:54:58.952 7f12cf1b74e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:58.952 7f12cf1b74e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:54:58.952 7f12cf1b74e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:54:58.952 7f12cf1b74e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:54:58.952 7f12cf1b74e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:54:58.953 7f12cf1b74e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:58.953 7f12cf1b74e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:54:58.953 7f12cf1b74e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:58.953 7f12cf1b74e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:54:58.953 7f12cf1b74e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.43:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:54:58.953 7f12cf1b74e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:02.953 7f12cf032198 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:55:02.954 7f12cf032198 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:55:02.954 7f12cf032198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:55:02.954 7f12cf032198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:02.954 7f12cf032198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:55:02.955 7f12cf032198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:02.955 7f12cf032198 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:55:02.955 7f12cf032198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:55:02.955 7f12cf032198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:02.955 7f12cf032198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:55:02.955 7f12cf032198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:02.955 7f12cf032198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:02.955 7f12cf032198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:55:02.955 7f12cf032198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:06.281 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2646-3'], 'obj_dict': {'bgp_router_parameters': {'address': '10.20.0.5', 'address_families': {'family': ['route-target', 'inet-vpn', 'e-vpn', 'erm-vpn', 'inet6-vpn']}, 'admin_down': False, 'auth_data': None, 'autonomous_system': 64512, 'cluster_id': None, 'gateway_address': None, 'hold_time': 0, 'identifier': '10.20.0.5', 'ipv6_gateway_address': None, 'local_autonomous_system': None, 'port': 179, 'router_type': 'control-node', 'source_port': None, 'vendor': 'contrail'}, 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2646-3', 'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2646-3'], 'id_perms': {'created': '2025-01-16T00:55:06.265337', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-01-16T00:55:06.265337', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9377276213756906292, 'uuid_mslong': 4513042020982999520}}, 'parent_type': 'routing-instance', 'parent_uuid': '821eadb7-8646-472f-b26e-133407e7cbf1', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '3ea18bce-951a-49e0-8222-c70891b34b34'}, 'oper': 'CREATE', 'request-id': 'req-84d53b58-27c7-4f19-887e-f5a444e3e73a', 'type': 'bgp_router', 'uuid': '3ea18bce-951a-49e0-8222-c70891b34b34'} 01/16/2025 00:55:06.343 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2646-2'], 'obj_dict': {'bgp_router_parameters': {'address': '10.20.0.6', 'address_families': {'family': ['route-target', 'inet-vpn', 'e-vpn', 'erm-vpn', 'inet6-vpn']}, 'admin_down': False, 'auth_data': None, 'autonomous_system': 64512, 'cluster_id': None, 'gateway_address': None, 'hold_time': 0, 'identifier': '10.20.0.6', 'ipv6_gateway_address': None, 'local_autonomous_system': None, 'port': 179, 'router_type': 'control-node', 'source_port': None, 'vendor': 'contrail'}, 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2646-2', 'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2646-2'], 'id_perms': {'created': '2025-01-16T00:55:06.324479', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-01-16T00:55:06.324479', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13307050869953845940, 'uuid_mslong': 2975605498777846027}}, 'parent_type': 'routing-instance', 'parent_uuid': '821eadb7-8646-472f-b26e-133407e7cbf1', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '294b796f-8a5a-410b-b8ac-24432fa6dab4'}, 'oper': 'CREATE', 'request-id': 'req-b4e9b9eb-6387-4fc8-9cd1-2d70ae4c1a6e', 'type': 'bgp_router', 'uuid': '294b796f-8a5a-410b-b8ac-24432fa6dab4'} 01/16/2025 00:55:06.371 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2646-1'], 'obj_dict': {'bgp_router_parameters': {'address': '10.20.0.27', 'address_families': {'family': ['route-target', 'inet-vpn', 'e-vpn', 'erm-vpn', 'inet6-vpn']}, 'admin_down': False, 'auth_data': None, 'autonomous_system': 64512, 'cluster_id': None, 'gateway_address': None, 'hold_time': 0, 'identifier': '10.20.0.27', 'ipv6_gateway_address': None, 'local_autonomous_system': None, 'port': 179, 'router_type': 'control-node', 'source_port': None, 'vendor': 'contrail'}, 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2646-1', 'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2646-1'], 'id_perms': {'created': '2025-01-16T00:55:06.345274', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-01-16T00:55:06.345274', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12004907447907900742, 'uuid_mslong': 7664401809683530921}}, 'parent_type': 'routing-instance', 'parent_uuid': '821eadb7-8646-472f-b26e-133407e7cbf1', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '6a5d6cd6-920c-44a9-a699-ffb819fddd46'}, 'oper': 'CREATE', 'request-id': 'req-29ac33a1-984f-4648-bed8-f29ae6707dbb', 'type': 'bgp_router', 'uuid': '6a5d6cd6-920c-44a9-a699-ffb819fddd46'} 01/16/2025 00:55:06.553 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-05c7da41-db12-4ed8-b32f-2ac98b334d9a', 'type': 'global_system_config', 'uuid': '22978d74-648d-4342-9707-c760fd7f98ed'} 01/16/2025 00:55:06.601 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-eee89e28-d769-42e7-a797-e657ca4ec184', 'type': 'global_system_config', 'uuid': '22978d74-648d-4342-9707-c760fd7f98ed'} 01/16/2025 00:55:06.674 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-ab025b30-85ee-48f5-93ff-f88bcc4db223', 'type': 'global_system_config', 'uuid': '22978d74-648d-4342-9707-c760fd7f98ed'} 01/16/2025 00:55:06.942 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2646-2'], 'oper': 'UPDATE', 'request-id': 'req-58135136-51ff-4fe6-a9b8-3b60596c422b', 'type': 'bgp_router', 'uuid': '294b796f-8a5a-410b-b8ac-24432fa6dab4'} 01/16/2025 00:55:06.952 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2646-3'], 'oper': 'UPDATE-IMPLICIT', 'request-id': 'req-58135136-51ff-4fe6-a9b8-3b60596c422b', 'type': 'bgp_router', 'uuid': '3ea18bce-951a-49e0-8222-c70891b34b34'} 01/16/2025 00:55:06.955 7f12ce79aac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:55:06.956 7f12ce79aac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:55:06.956 7f12ce79aac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:55:06.956 7f12ce79aac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:06.957 7f12ce79aac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:55:06.957 7f12ce79aac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:06.957 7f12ce79aac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:55:06.957 7f12ce79aac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:55:06.957 7f12ce79aac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:06.957 7f12ce79aac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:55:06.957 7f12ce79aac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:06.957 7f12ce79aac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:06.957 7f12ce79aac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.29:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:55:06.957 7f12ce79aac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:07.066 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2646-1'], 'oper': 'UPDATE', 'request-id': 'req-22862af9-ff2e-48f1-b3c2-3530f10cefd6', 'type': 'bgp_router', 'uuid': '6a5d6cd6-920c-44a9-a699-ffb819fddd46'} 01/16/2025 00:55:07.072 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2646-3'], 'oper': 'UPDATE-IMPLICIT', 'request-id': 'req-22862af9-ff2e-48f1-b3c2-3530f10cefd6', 'type': 'bgp_router', 'uuid': '3ea18bce-951a-49e0-8222-c70891b34b34'} 01/16/2025 00:55:07.126 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2646-1'], 'oper': 'UPDATE', 'request-id': 'req-2d177cfb-1ba7-41dd-a53e-13ecd5176d64', 'type': 'bgp_router', 'uuid': '6a5d6cd6-920c-44a9-a699-ffb819fddd46'} 01/16/2025 00:55:07.134 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2646-2'], 'oper': 'UPDATE-IMPLICIT', 'request-id': 'req-2d177cfb-1ba7-41dd-a53e-13ecd5176d64', 'type': 'bgp_router', 'uuid': '294b796f-8a5a-410b-b8ac-24432fa6dab4'} 01/16/2025 00:55:10.957 7f12cdc8ad68 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:55:10.959 7f12cdc8ad68 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:55:10.959 7f12cdc8ad68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:55:10.959 7f12cdc8ad68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:10.959 7f12cdc8ad68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:55:10.959 7f12cdc8ad68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:10.959 7f12cdc8ad68 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:55:10.960 7f12cdc8ad68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:55:10.960 7f12cdc8ad68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:10.960 7f12cdc8ad68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:55:10.960 7f12cdc8ad68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:10.960 7f12cdc8ad68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:10.960 7f12cdc8ad68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.43:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:55:10.960 7f12cdc8ad68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:14.960 7f12cdc8ad68 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:55:14.961 7f12cdc8ad68 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:55:14.961 7f12cdc8ad68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:55:14.961 7f12cdc8ad68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:14.961 7f12cdc8ad68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:55:14.961 7f12cdc8ad68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:14.961 7f12cdc8ad68 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:55:14.962 7f12cdc8ad68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:55:14.962 7f12cdc8ad68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:14.962 7f12cdc8ad68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:55:14.962 7f12cdc8ad68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:14.962 7f12cdc8ad68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:14.962 7f12cdc8ad68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:55:14.962 7f12cdc8ad68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:18.962 7f12cdc8ad68 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:55:18.963 7f12cdc8ad68 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:55:18.963 7f12cdc8ad68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:55:18.964 7f12cdc8ad68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:18.964 7f12cdc8ad68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:55:18.964 7f12cdc8ad68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:18.964 7f12cdc8ad68 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:55:18.964 7f12cdc8ad68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:55:18.964 7f12cdc8ad68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:18.964 7f12cdc8ad68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:55:18.965 7f12cdc8ad68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:18.965 7f12cdc8ad68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:18.965 7f12cdc8ad68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.29:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:55:18.965 7f12cdc8ad68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:22.964 7f12cdc8ad68 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:55:22.966 7f12cdc8ad68 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:55:22.966 7f12cdc8ad68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:55:22.966 7f12cdc8ad68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:22.967 7f12cdc8ad68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:55:22.967 7f12cdc8ad68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:22.967 7f12cdc8ad68 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:55:22.967 7f12cdc8ad68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:55:22.967 7f12cdc8ad68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:22.968 7f12cdc8ad68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:55:22.968 7f12cdc8ad68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:22.968 7f12cdc8ad68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:22.968 7f12cdc8ad68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.43:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:55:22.968 7f12cdc8ad68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:26.966 7f12cdc8ad68 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:55:26.968 7f12cdc8ad68 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:55:26.968 7f12cdc8ad68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:55:26.968 7f12cdc8ad68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:26.968 7f12cdc8ad68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:55:26.968 7f12cdc8ad68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:26.968 7f12cdc8ad68 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:55:26.969 7f12cdc8ad68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:55:26.969 7f12cdc8ad68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:26.969 7f12cdc8ad68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:55:26.969 7f12cdc8ad68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:26.969 7f12cdc8ad68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:26.969 7f12cdc8ad68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:55:26.969 7f12cdc8ad68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:30.968 7f12cff5e390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:55:30.970 7f12cff5e390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:55:30.970 7f12cff5e390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:55:30.970 7f12cff5e390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:30.971 7f12cff5e390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:55:30.971 7f12cff5e390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:30.971 7f12cff5e390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:55:30.971 7f12cff5e390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:55:30.971 7f12cff5e390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:30.971 7f12cff5e390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:55:30.971 7f12cff5e390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:30.972 7f12cff5e390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:30.972 7f12cff5e390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.29:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:55:30.972 7f12cff5e390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:34.971 7f12cff5e390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:55:34.973 7f12cff5e390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:55:34.973 7f12cff5e390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:55:34.973 7f12cff5e390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:34.973 7f12cff5e390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:55:34.973 7f12cff5e390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:34.973 7f12cff5e390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:55:34.973 7f12cff5e390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:55:34.974 7f12cff5e390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:34.974 7f12cff5e390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:55:34.974 7f12cff5e390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:34.974 7f12cff5e390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:34.974 7f12cff5e390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.43:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:55:34.974 7f12cff5e390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:38.974 7f12cff5e390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:55:38.975 7f12cff5e390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:55:38.975 7f12cff5e390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:55:38.975 7f12cff5e390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:38.975 7f12cff5e390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:55:38.975 7f12cff5e390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:38.975 7f12cff5e390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:55:38.976 7f12cff5e390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:55:38.976 7f12cff5e390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:38.976 7f12cff5e390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:55:38.976 7f12cff5e390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:38.976 7f12cff5e390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:38.976 7f12cff5e390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:55:38.976 7f12cff5e390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:42.976 7f12cdc828d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:55:42.978 7f12cdc828d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:55:42.978 7f12cdc828d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:55:42.978 7f12cdc828d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:42.978 7f12cdc828d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:55:42.978 7f12cdc828d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:42.978 7f12cdc828d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:55:42.979 7f12cdc828d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:55:42.979 7f12cdc828d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:42.979 7f12cdc828d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:55:42.979 7f12cdc828d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:42.979 7f12cdc828d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:42.979 7f12cdc828d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.29:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:55:42.979 7f12cdc828d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:46.978 7f12cdc828d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:55:46.980 7f12cdc828d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:55:46.980 7f12cdc828d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:55:46.980 7f12cdc828d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:46.980 7f12cdc828d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:55:46.980 7f12cdc828d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:46.980 7f12cdc828d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:55:46.981 7f12cdc828d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:55:46.981 7f12cdc828d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:46.981 7f12cdc828d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:55:46.981 7f12cdc828d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:46.981 7f12cdc828d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:46.981 7f12cdc828d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.43:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:55:46.981 7f12cdc828d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:50.981 7f12cdc828d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:55:50.981 7f12cdc828d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:55:50.982 7f12cdc828d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:55:50.982 7f12cdc828d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:50.982 7f12cdc828d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:55:50.982 7f12cdc828d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:50.982 7f12cdc828d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:55:50.982 7f12cdc828d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:55:50.982 7f12cdc828d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:50.982 7f12cdc828d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:55:50.982 7f12cdc828d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:50.983 7f12cdc828d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:50.983 7f12cdc828d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:55:50.983 7f12cdc828d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:54.982 7f12cdc828d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:55:54.984 7f12cdc828d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:55:54.984 7f12cdc828d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:55:54.984 7f12cdc828d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:54.985 7f12cdc828d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:55:54.985 7f12cdc828d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:54.985 7f12cdc828d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:55:54.985 7f12cdc828d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:55:54.985 7f12cdc828d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:54.985 7f12cdc828d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:55:54.985 7f12cdc828d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:54.985 7f12cdc828d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:54.985 7f12cdc828d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.29:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:55:54.986 7f12cdc828d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:58.985 7f12cdc828d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:55:58.987 7f12cdc828d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:55:58.987 7f12cdc828d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:55:58.987 7f12cdc828d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:58.987 7f12cdc828d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:55:58.987 7f12cdc828d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:55:58.987 7f12cdc828d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:55:58.988 7f12cdc828d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:55:58.988 7f12cdc828d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:58.988 7f12cdc828d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:55:58.988 7f12cdc828d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:58.988 7f12cdc828d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:55:58.988 7f12cdc828d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.43:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:55:58.988 7f12cdc828d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:02.988 7f12cdc0e0f0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:56:02.989 7f12cdc0e0f0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:56:02.990 7f12cdc0e0f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:56:02.990 7f12cdc0e0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:02.990 7f12cdc0e0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:56:02.990 7f12cdc0e0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:02.990 7f12cdc0e0f0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:56:02.990 7f12cdc0e0f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:56:02.990 7f12cdc0e0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:02.991 7f12cdc0e0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:56:02.991 7f12cdc0e0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:02.991 7f12cdc0e0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:02.991 7f12cdc0e0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:56:02.991 7f12cdc0e0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:06.990 7f12cdc0e0f0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:56:06.992 7f12cdc0e0f0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:56:06.992 7f12cdc0e0f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:56:06.992 7f12cdc0e0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:06.993 7f12cdc0e0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:56:06.993 7f12cdc0e0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:06.993 7f12cdc0e0f0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:56:06.993 7f12cdc0e0f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:56:06.993 7f12cdc0e0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:06.994 7f12cdc0e0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:56:06.994 7f12cdc0e0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:06.994 7f12cdc0e0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:06.994 7f12cdc0e0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.29:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:56:06.994 7f12cdc0e0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:10.993 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:56:10.995 7f12cdc6ff60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:56:10.995 7f12cdc6ff60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:56:10.995 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:10.995 7f12cdc6ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:56:10.995 7f12cdc6ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:10.995 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:56:10.996 7f12cdc6ff60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:56:10.996 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:10.996 7f12cdc6ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:56:10.996 7f12cdc6ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:10.996 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:10.996 7f12cdc6ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.43:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:56:10.996 7f12cdc6ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:14.905 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2646-3.'], 'obj_dict': {'database_node_ip_address': '10.0.0.23', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2646-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2646-3.'], 'id_perms': {'created': '2025-01-16T00:56:14.880542', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-01-16T00:56:14.880542', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10388389068655718969, 'uuid_mslong': 16951382534951946975}}, 'parent_type': 'global-system-config', 'parent_uuid': '22978d74-648d-4342-9707-c760fd7f98ed', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'eb3f689a-6f92-4adf-902a-f8d161c67239'}, 'oper': 'CREATE', 'request-id': 'req-34fc1c22-0ca1-48cb-86f3-82fff70161ac', 'type': 'database_node', 'uuid': 'eb3f689a-6f92-4adf-902a-f8d161c67239'} 01/16/2025 00:56:14.996 7f12cdba1240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:56:14.996 7f12cdba1240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:56:14.997 7f12cdba1240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:56:14.997 7f12cdba1240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:14.997 7f12cdba1240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:56:14.997 7f12cdba1240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:14.997 7f12cdba1240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:56:14.997 7f12cdba1240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:56:14.997 7f12cdba1240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:14.998 7f12cdba1240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:56:14.998 7f12cdba1240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:14.998 7f12cdba1240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:14.998 7f12cdba1240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:56:14.998 7f12cdba1240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:15.010 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2646-1.'], 'obj_dict': {'database_node_ip_address': '10.0.0.43', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2646-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2646-1.'], 'id_perms': {'created': '2025-01-16T00:56:14.976597', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-01-16T00:56:14.976597', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12367999836824040159, 'uuid_mslong': 15544294456786371762}}, 'parent_type': 'global-system-config', 'parent_uuid': '22978d74-648d-4342-9707-c760fd7f98ed', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'd7b86d73-84d3-44b2-aba3-f652b7fc72df'}, 'oper': 'CREATE', 'request-id': 'req-371e9436-e3f7-48ac-80d3-b10501d424f0', 'type': 'database_node', 'uuid': 'd7b86d73-84d3-44b2-aba3-f652b7fc72df'} 01/16/2025 00:56:15.032 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2646-2.'], 'obj_dict': {'database_node_ip_address': '10.0.0.29', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2646-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2646-2.'], 'id_perms': {'created': '2025-01-16T00:56:14.988452', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-01-16T00:56:14.988452', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11526295004259956339, 'uuid_mslong': 15757270789541152174}}, 'parent_type': 'global-system-config', 'parent_uuid': '22978d74-648d-4342-9707-c760fd7f98ed', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'daad124c-283d-45ae-9ff5-a03cc9463673'}, 'oper': 'CREATE', 'request-id': 'req-ae66d52a-7317-4bc2-a213-9f6ef3b71a8e', 'type': 'database_node', 'uuid': 'daad124c-283d-45ae-9ff5-a03cc9463673'} 01/16/2025 00:56:18.997 7f12cdc8c2e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:56:18.999 7f12cdc8c2e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:56:18.999 7f12cdc8c2e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:56:19.000 7f12cdc8c2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:19.000 7f12cdc8c2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:56:19.000 7f12cdc8c2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:19.000 7f12cdc8c2e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:56:19.000 7f12cdc8c2e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:56:19.000 7f12cdc8c2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:19.001 7f12cdc8c2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:56:19.001 7f12cdc8c2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:19.001 7f12cdc8c2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:19.001 7f12cdc8c2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.29:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:56:19.001 7f12cdc8c2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:23.000 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:56:23.003 7f12cdc6ff60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:56:23.003 7f12cdc6ff60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:56:23.003 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:23.003 7f12cdc6ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:56:23.003 7f12cdc6ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:23.003 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:56:23.004 7f12cdc6ff60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:56:23.004 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:23.004 7f12cdc6ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:56:23.004 7f12cdc6ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:23.004 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:23.004 7f12cdc6ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.43:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:56:23.004 7f12cdc6ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:27.004 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:56:27.005 7f12cdc6ff60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:56:27.005 7f12cdc6ff60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:56:27.005 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:27.006 7f12cdc6ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:56:27.006 7f12cdc6ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:27.006 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:56:27.006 7f12cdc6ff60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:56:27.006 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:27.006 7f12cdc6ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:56:27.007 7f12cdc6ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:27.007 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:27.007 7f12cdc6ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:56:27.007 7f12cdc6ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:31.006 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:56:31.008 7f12cdc6ff60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:56:31.009 7f12cdc6ff60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:56:31.009 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:31.009 7f12cdc6ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:56:31.009 7f12cdc6ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:31.009 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:56:31.009 7f12cdc6ff60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:56:31.009 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:31.010 7f12cdc6ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:56:31.010 7f12cdc6ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:31.010 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:31.010 7f12cdc6ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.29:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:56:31.010 7f12cdc6ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:35.009 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:56:35.010 7f12cdc6ff60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:56:35.011 7f12cdc6ff60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:56:35.011 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:35.011 7f12cdc6ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:56:35.011 7f12cdc6ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:35.011 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:56:35.012 7f12cdc6ff60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:56:35.012 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:35.012 7f12cdc6ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:56:35.012 7f12cdc6ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:35.012 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:35.012 7f12cdc6ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.43:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:56:35.012 7f12cdc6ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:39.011 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:56:39.013 7f12cdc6ff60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:56:39.013 7f12cdc6ff60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:56:39.013 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:39.013 7f12cdc6ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:56:39.013 7f12cdc6ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:39.013 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:56:39.014 7f12cdc6ff60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:56:39.014 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:39.014 7f12cdc6ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:56:39.014 7f12cdc6ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:39.014 7f12cdc6ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:39.014 7f12cdc6ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:56:39.014 7f12cdc6ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:43.013 7f12cdbad8d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:56:43.015 7f12cdbad8d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:56:43.015 7f12cdbad8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:56:43.015 7f12cdbad8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:43.015 7f12cdbad8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:56:43.015 7f12cdbad8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:43.015 7f12cdbad8d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:56:43.016 7f12cdbad8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:56:43.016 7f12cdbad8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:43.016 7f12cdbad8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:56:43.016 7f12cdbad8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:43.016 7f12cdbad8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:43.016 7f12cdbad8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.29:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:56:43.016 7f12cdbad8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:47.015 7f12cdbad8d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:56:47.017 7f12cdbad8d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:56:47.017 7f12cdbad8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:56:47.017 7f12cdbad8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:47.018 7f12cdbad8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:56:47.018 7f12cdbad8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:47.018 7f12cdbad8d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:56:47.018 7f12cdbad8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:56:47.018 7f12cdbad8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:47.018 7f12cdbad8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:56:47.018 7f12cdbad8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:47.018 7f12cdbad8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:47.018 7f12cdbad8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.43:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:56:47.018 7f12cdbad8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:51.018 7f12cdbad8d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:56:51.019 7f12cdbad8d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:56:51.019 7f12cdbad8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:56:51.020 7f12cdbad8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:51.020 7f12cdbad8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:56:51.020 7f12cdbad8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:51.020 7f12cdbad8d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:56:51.020 7f12cdbad8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:56:51.020 7f12cdbad8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:51.020 7f12cdbad8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:56:51.020 7f12cdbad8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:51.021 7f12cdbad8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:51.021 7f12cdbad8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:56:51.021 7f12cdbad8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:55.021 7f12cdbad8d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:56:55.022 7f12cdbad8d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:56:55.023 7f12cdbad8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:56:55.023 7f12cdbad8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:55.023 7f12cdbad8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:56:55.023 7f12cdbad8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:55.023 7f12cdbad8d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:56:55.023 7f12cdbad8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:56:55.024 7f12cdbad8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:55.024 7f12cdbad8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:56:55.024 7f12cdbad8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:55.024 7f12cdbad8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:55.024 7f12cdbad8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.29:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:56:55.024 7f12cdbad8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:59.023 7f12cdbad8d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:56:59.025 7f12cdbad8d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:56:59.025 7f12cdbad8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:56:59.025 7f12cdbad8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:59.025 7f12cdbad8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:56:59.025 7f12cdbad8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:56:59.025 7f12cdbad8d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:56:59.026 7f12cdbad8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:56:59.026 7f12cdbad8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:59.026 7f12cdbad8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:56:59.026 7f12cdbad8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:59.026 7f12cdbad8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:56:59.026 7f12cdbad8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.43:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:56:59.026 7f12cdbad8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:57:03.026 7f12cdc13780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:57:03.027 7f12cdc13780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:57:03.027 7f12cdc13780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:57:03.027 7f12cdc13780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:57:03.027 7f12cdc13780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:57:03.027 7f12cdc13780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:57:03.027 7f12cdc13780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:57:03.028 7f12cdc13780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:57:03.028 7f12cdc13780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:57:03.028 7f12cdc13780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:57:03.028 7f12cdc13780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:57:03.028 7f12cdc13780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:57:03.028 7f12cdc13780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:57:03.028 7f12cdc13780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:57:07.027 7f12cdc13780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:57:07.029 7f12cdc13780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:57:07.029 7f12cdc13780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:57:07.029 7f12cdc13780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:57:07.029 7f12cdc13780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:57:07.029 7f12cdc13780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:57:07.030 7f12cdc13780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:57:07.030 7f12cdc13780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:57:07.030 7f12cdc13780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:57:07.030 7f12cdc13780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:57:07.030 7f12cdc13780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:57:07.030 7f12cdc13780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:57:07.030 7f12cdc13780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.29:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:57:07.030 7f12cdc13780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:57:11.030 7f12cdc7c630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:57:11.032 7f12cdc7c630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 01/16/2025 00:57:11.032 7f12cdc7c630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:57:11.032 7f12cdc7c630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:57:11.032 7f12cdc7c630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:57:11.033 7f12cdc7c630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:57:11.033 7f12cdc7c630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/16/2025 00:57:11.033 7f12cdc7c630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/16/2025 00:57:11.033 7f12cdc7c630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:57:11.033 7f12cdc7c630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.43:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/16/2025 00:57:11.033 7f12cdc7c630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:57:11.034 7f12cdc7c630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:57:11.034 7f12cdc7c630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.43:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:57:11.034 7f12cdc7c630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:57:15.033 7f12cf1baeb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:57:15.035 7f12cf1baeb8 [contrail-device-manager] [INFO]: Session Event: TCP Connected 01/16/2025 00:57:15.035 7f12cf1baeb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:57:15.035 7f12cf1baeb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:57:15.035 7f12cf1baeb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:57:15.035 7f12cf1baeb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:57:15.035 7f12cf1baeb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect] 01/16/2025 00:57:15.036 7f12cf1baeb8 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19 01/16/2025 00:57:15.036 7f12cf1baeb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 01/16/2025 00:57:15.036 7f12cf1baeb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 01/16/2025 00:57:15.036 7f12cf1baeb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 01/16/2025 00:57:15.036 7f12cdc00a20 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 01/16/2025 00:57:15.037 7f12cdc00a20 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 01/16/2025 00:57:17.163 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2646-2.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.29', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2646-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2646-2.'], 'id_perms': {'created': '2025-01-16T00:57:17.139080', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-01-16T00:57:17.139080', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12860226364564638075, 'uuid_mslong': 3152872497109551074}}, 'parent_type': 'global-system-config', 'parent_uuid': '22978d74-648d-4342-9707-c760fd7f98ed', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '2bc140d4-dc50-47e2-b278-b3bdfefd097b'}, 'oper': 'CREATE', 'request-id': 'req-2c48b82b-224b-417c-8797-f8ad7cc03d9b', 'type': 'analytics_node', 'uuid': '2bc140d4-dc50-47e2-b278-b3bdfefd097b'} 01/16/2025 00:57:17.185 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2646-1.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.43', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2646-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2646-1.'], 'id_perms': {'created': '2025-01-16T00:57:17.149121', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-01-16T00:57:17.149121', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12989567633272752670, 'uuid_mslong': 14597711981351747642}}, 'parent_type': 'global-system-config', 'parent_uuid': '22978d74-648d-4342-9707-c760fd7f98ed', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'ca957db4-7def-403a-b444-36f0d7224a1e'}, 'oper': 'CREATE', 'request-id': 'req-df52b944-9838-4ef7-9ab0-2bf3e82f370e', 'type': 'analytics_node', 'uuid': 'ca957db4-7def-403a-b444-36f0d7224a1e'} 01/16/2025 00:57:17.211 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2646-3.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.23', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2646-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2646-3.'], 'id_perms': {'created': '2025-01-16T00:57:17.179925', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-01-16T00:57:17.179925', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13083894411625429544, 'uuid_mslong': 17298524979550307307}}, 'parent_type': 'global-system-config', 'parent_uuid': '22978d74-648d-4342-9707-c760fd7f98ed', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'f010b4d1-4051-43eb-b593-54a55ba6fa28'}, 'oper': 'CREATE', 'request-id': 'req-3a728b7d-7eab-4e7f-9af0-01ae5c39d25d', 'type': 'analytics_node', 'uuid': 'f010b4d1-4051-43eb-b593-54a55ba6fa28'} 01/16/2025 00:57:28.162 7f12cff59d68 [contrail-device-manager] [ERROR]: Session Event: TCP Connection Closed 01/16/2025 00:57:28.162 7f12cde7e2e8 [contrail-device-manager] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 01/16/2025 00:57:28.163 7f12cde7e2e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 01/16/2025 00:57:28.163 7f12cde7e2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:57:28.163 7f12cde7e2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 01/16/2025 00:57:28.163 7f12cde7e2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:57:28.163 7f12cde7e2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:57:28.163 7f12cde7e2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 1 pid = 140 http_port = 8096 start_time = 1736988778883731 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.29:8086, 10.0.0.43:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/16/2025 00:57:28.163 7f12cde7e2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/16/2025 00:57:32.164 7f12cde7e2e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/16/2025 00:57:32.166 7f12cde7e2e8 [contrail-device-manager] [INFO]: Session Event: TCP Connected 01/16/2025 00:57:32.166 7f12cde7e2e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/16/2025 00:57:32.166 7f12cde7e2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:57:32.166 7f12cde7e2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2646-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.29:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.43:5673, 10.0.0.29:5673, 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.43:9041, 10.0.0.29:9041, 10.0.0.23:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.43:2181, 10.0.0.29:2181, 10.0.0.23:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.43,10.0.0.29,10.0.0.23:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/16/2025 00:57:32.167 7f12cde7e2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/16/2025 00:57:32.167 7f12cde7e2e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect] 01/16/2025 00:57:32.167 7f12cde7e2e8 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19 01/16/2025 00:57:32.167 7f12cde7e2e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 01/16/2025 00:57:32.167 7f12cde7e2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 01/16/2025 00:57:32.167 7f12cde7e2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 01/16/2025 00:57:32.167 7f12cff5e048 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 01/16/2025 00:57:32.168 7f12cff5e048 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 01/16/2025 00:57:32.186 7f12cff51eb8 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001017 01/16/2025 00:57:32.187 7f12cff51eb8 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 01/16/2025 00:57:32.187 7f12d4d9a6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 01/16/2025 00:57:32.187 7f12d4d9a6d8 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 0 01/16/2025 00:57:32.187 7f12d4d9a6d8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 00:57:32.187 7f12d4d9a6d8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 01/16/2025 00:57:32.188 7f12d4d9a6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 01/16/2025 00:57:32.188 7f12cff5e048 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 01/16/2025 00:57:32.189 7f12cff5e048 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 01/16/2025 00:57:39.401 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2646-1.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.43', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2646-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2646-1.'], 'id_perms': {'created': '2025-01-16T00:57:39.379740', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-01-16T00:57:39.379740', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13401031202637362395, 'uuid_mslong': 1255001488023768173}}, 'parent_type': 'global-system-config', 'parent_uuid': '22978d74-648d-4342-9707-c760fd7f98ed', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '116aa934-3e3e-4c6d-b9fa-06e0892034db'}, 'oper': 'CREATE', 'request-id': 'req-0ff2ba28-ee17-45f4-9b4f-74ade02282be', 'type': 'analytics_alarm_node', 'uuid': '116aa934-3e3e-4c6d-b9fa-06e0892034db'} 01/16/2025 00:57:39.458 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2646-2.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.29', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2646-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2646-2.'], 'id_perms': {'created': '2025-01-16T00:57:39.404685', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-01-16T00:57:39.404685', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11898687839266433963, 'uuid_mslong': 1592637898675929707}}, 'parent_type': 'global-system-config', 'parent_uuid': '22978d74-648d-4342-9707-c760fd7f98ed', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '161a2fbb-0e67-426b-a520-a18c40bc6bab'}, 'oper': 'CREATE', 'request-id': 'req-580cda83-4f60-4616-8424-5ff8c44d1128', 'type': 'analytics_alarm_node', 'uuid': '161a2fbb-0e67-426b-a520-a18c40bc6bab'} 01/16/2025 00:57:39.548 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2646-3.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.23', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2646-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2646-3.'], 'id_perms': {'created': '2025-01-16T00:57:39.511831', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-01-16T00:57:39.511831', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13612157922925712275, 'uuid_mslong': 6651862862675264300}}, 'parent_type': 'global-system-config', 'parent_uuid': '22978d74-648d-4342-9707-c760fd7f98ed', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '5c502a07-d05f-432c-bce8-1982b6819393'}, 'oper': 'CREATE', 'request-id': 'req-af307334-d52a-46a7-a5f5-7d8a2b2c334d', 'type': 'analytics_alarm_node', 'uuid': '5c502a07-d05f-432c-bce8-1982b6819393'} 01/16/2025 00:57:43.057 7f12cff51eb8 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001297 01/16/2025 00:57:43.057 7f12cff51eb8 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 01/16/2025 00:57:43.058 7f12cff51c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 01/16/2025 00:57:43.058 7f12cff51c18 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 2 01/16/2025 00:57:43.058 7f12cff51c18 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 00:57:43.058 7f12cff51c18 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 01/16/2025 00:57:53.899 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2646-1.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.43', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2646-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2646-1.'], 'id_perms': {'created': '2025-01-16T00:57:53.864373', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-01-16T00:57:53.864373', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12886924083119876159, 'uuid_mslong': 5763311014538200173}}, 'parent_type': 'global-system-config', 'parent_uuid': '22978d74-648d-4342-9707-c760fd7f98ed', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '4ffb64d5-17a5-446d-b2d7-8d2cfdc9683f'}, 'oper': 'CREATE', 'request-id': 'req-48f7362e-6607-4810-a890-b071876681b9', 'type': 'analytics_snmp_node', 'uuid': '4ffb64d5-17a5-446d-b2d7-8d2cfdc9683f'} 01/16/2025 00:57:54.435 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2646-3.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.23', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2646-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2646-3.'], 'id_perms': {'created': '2025-01-16T00:57:54.411359', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-01-16T00:57:54.411359', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12276455036309359186, 'uuid_mslong': 6048722767998701149}}, 'parent_type': 'global-system-config', 'parent_uuid': '22978d74-648d-4342-9707-c760fd7f98ed', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '53f1614f-5b61-4e5d-aa5e-bacfe4464a52'}, 'oper': 'CREATE', 'request-id': 'req-b515dcb3-2ae1-465f-80d6-e66c2bce4808', 'type': 'analytics_snmp_node', 'uuid': '53f1614f-5b61-4e5d-aa5e-bacfe4464a52'} 01/16/2025 00:57:54.527 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2646-2.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.29', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2646-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2646-2.'], 'id_perms': {'created': '2025-01-16T00:57:54.499110', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-01-16T00:57:54.499110', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11101887040193823160, 'uuid_mslong': 14769472425524807280}}, 'parent_type': 'global-system-config', 'parent_uuid': '22978d74-648d-4342-9707-c760fd7f98ed', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'ccf7b4eb-4355-4670-9a11-d371574401b8'}, 'oper': 'CREATE', 'request-id': 'req-50b90e50-e4f3-4c71-8814-c5b13eaee0bf', 'type': 'analytics_snmp_node', 'uuid': 'ccf7b4eb-4355-4670-9a11-d371574401b8'} 01/16/2025 00:59:54.463 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-f39e9f6e-50f7-4d65-9d12-1d7b8c2852dd', 'type': 'global_system_config', 'uuid': '22978d74-648d-4342-9707-c760fd7f98ed'} 01/16/2025 00:59:54.611 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2646-2'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-2646-2', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2646-2'], 'id_perms': {'created': '2025-01-16T00:59:54.597365', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-01-16T00:59:54.597365', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9508196493153674946, 'uuid_mslong': 7812083512600185300}}, 'parent_type': 'global-system-config', 'parent_uuid': '22978d74-648d-4342-9707-c760fd7f98ed', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '6c6a1890-8438-49d4-83f3-e6557d0a26c2', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.24'}, 'oper': 'CREATE', 'request-id': 'req-c3a21e95-ad50-4e8b-a4e8-cfa565deef0a', 'type': 'virtual_router', 'uuid': '6c6a1890-8438-49d4-83f3-e6557d0a26c2'} 01/16/2025 00:59:54.976 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2646-2', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2646-2', 'vhost0'], 'id_perms': {'created': '2025-01-16T00:59:54.954527', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-01-16T00:59:54.954527', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13335249911922711153, 'uuid_mslong': 14007366754167898567}}, 'parent_type': 'virtual-router', 'parent_uuid': '6c6a1890-8438-49d4-83f3-e6557d0a26c2', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'c26429d9-44cb-41c7-b910-532447365a71', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:c2:64:29:d9:44']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': '0678c6bd-5bfd-4b8a-8d00-0b438b2f5966'}]}, 'oper': 'CREATE', 'request-id': 'req-68f304e7-31d9-42bb-a97b-db5f0cc286ee', 'type': 'virtual_machine_interface', 'uuid': 'c26429d9-44cb-41c7-b910-532447365a71'} 01/16/2025 00:59:55.048 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2646-2', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-68f304e7-31d9-42bb-a97b-db5f0cc286ee', 'type': 'virtual_machine_interface', 'uuid': 'c26429d9-44cb-41c7-b910-532447365a71'} 01/16/2025 00:59:55.564 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-ffd9618e-80d9-485c-a612-3297231af9b5', 'type': 'access_control_list', 'uuid': '1f1d2317-920f-473a-978b-e0a09ae1808d'} 01/16/2025 01:00:46.827 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2646-1'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-2646-1', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2646-1'], 'id_perms': {'created': '2025-01-16T01:00:46.808929', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-01-16T01:00:46.808929', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11648297938632170036, 'uuid_mslong': 10690037606337167446}}, 'parent_type': 'global-system-config', 'parent_uuid': '22978d74-648d-4342-9707-c760fd7f98ed', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '945aa490-914b-4056-a1a7-11428615da34', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.8'}, 'oper': 'CREATE', 'request-id': 'req-ae8fb6da-9290-49c9-a854-b3c1e1bc71c8', 'type': 'virtual_router', 'uuid': '945aa490-914b-4056-a1a7-11428615da34'} 01/16/2025 01:00:47.169 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2646-1', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2646-1', 'vhost0'], 'id_perms': {'created': '2025-01-16T01:00:47.145756', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-01-16T01:00:47.145756', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13224542864807814971, 'uuid_mslong': 16058644606708106739}}, 'parent_type': 'virtual-router', 'parent_uuid': '945aa490-914b-4056-a1a7-11428615da34', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'dedbc42f-fd14-45f3-b787-03ab3dcbe73b', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:de:db:c4:2f:fd']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': '0678c6bd-5bfd-4b8a-8d00-0b438b2f5966'}]}, 'oper': 'CREATE', 'request-id': 'req-e1464bd0-69cc-4fa2-9657-bd62b578dbc6', 'type': 'virtual_machine_interface', 'uuid': 'dedbc42f-fd14-45f3-b787-03ab3dcbe73b'} 01/16/2025 01:00:47.234 7f12cdd1d390 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2646-1', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-e1464bd0-69cc-4fa2-9657-bd62b578dbc6', 'type': 'virtual_machine_interface', 'uuid': 'dedbc42f-fd14-45f3-b787-03ab3dcbe73b'} 01/16/2025 01:01:02.776 7f12cfeded68 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:01:57.484 7f12cdd1dc18 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:02:20.323 7f12d6beea20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:02:42.758 7f12cdc73d68 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:03:06.030 7f12cdc73d68 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:03:28.638 7f12cdc78c18 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:03:50.475 7f12cdd26198 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:04:13.221 7f12cde7e4e0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:04:36.641 7f12cdb866d8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:04:58.711 7f12cddac048 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:05:21.201 7f12cde7e2e8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:05:44.525 7f12cdc0b780 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:06:06.477 7f12d4d7cac8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:06:29.074 7f12cff5e438 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:06:52.131 7f12cdc9a978 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:07:14.296 7f12cdc78ac8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:07:37.032 7f12d4c92630 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:07:59.775 7f12cdbff4e0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:08:22.482 7f12cdbff4e0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:08:45.086 7f12cdec1ac8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:09:07.813 7f12cde7e2e8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:09:31.157 7f12cdc78c18 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:09:53.128 7f12cfeded68 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:10:15.852 7f12cdbff438 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:10:39.541 7f12cdbff438 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:10:51.876 7f12cdc0b780 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:11:04.782 7f12cfef9780 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:11:17.606 7f12d6beea20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:11:30.258 7f12cf3ab8d0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:11:42.505 7f12cf1baeb8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:12:06.237 7f12cf01bf60 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:12:29.169 7f12d4c920f0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:12:51.070 7f12cf037eb8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:13:14.602 7f12cdb86978 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:13:37.174 7f12d56a7828 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:13:58.841 7f12cff5e048 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:14:21.991 7f12cff59cc0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:14:44.117 7f12cdc78ac8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:15:06.150 7f12cdbff780 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:15:29.759 7f12cf01bf60 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:15:51.718 7f12cdbff4e0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:16:13.749 7f12cdc0bf60 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:16:37.354 7f12cfede6d8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:16:59.734 7f12cfef9780 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:17:22.157 7f12d4e0eac8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:17:45.713 7f12d4c92780 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:18:08.586 7f12d4c92780 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:18:31.213 7f12cdc78ac8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:18:54.635 7f12cfeded68 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:19:17.655 7f12cdc0be10 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:19:39.968 7f12cdc73d68 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:20:03.365 7f12cdc9a6d8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:20:25.702 7f12d4d7cac8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:20:47.692 7f12d4d7cac8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:21:10.777 7f12cff5e390 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:21:33.810 7f12cdec1ac8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:21:55.543 7f12cdc0b780 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:22:18.281 7f12cdbff4e0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:22:40.978 7f12d4d9a6d8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:23:03.012 7f12d4d9a6d8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:23:26.082 7f12cdbadf60 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:23:38.095 7f12cde7e2e8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:23:48.823 7f12cdbffd68 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 01/16/2025 01:24:42.995 7f12cdc0bb70 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map