05/26/2025 00:44:15.888 7faaf29b9c08 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 05/26/2025 00:44:15.888 7faaf29b9c08 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 05/26/2025 00:44:15.888 7faaf29b9c08 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 05/26/2025 00:44:15.888 7faaea0b52e8 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 05/26/2025 00:44:15.889 7faaea0b52e8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 05/26/2025 00:44:15.889 7faaea0b52e8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 05/26/2025 00:44:15.890 7faaea0b52e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 05/26/2025 00:44:15.890 7faaea0b52e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:15.890 7faaea0b52e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 05/26/2025 00:44:15.890 7faaea0b52e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:15.890 7faaea0b52e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:15.890 7faaea0b52e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 start_time = 1748220255889771 collector_name = collector_ip = collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 05/26/2025 00:44:15.890 7faaea0b52e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:15.936 7faaf29b9c08 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.19:5673//', 'pyamqp://guest:guest@10.0.0.41:5673//', 'pyamqp://guest:guest@10.0.0.61:5673//'] 05/26/2025 00:44:15.944 7faaf29b9c08 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 05/26/2025 00:44:15.945 7faaea005588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:15.945 7faaea005588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 05/26/2025 00:44:15.945 7faaea005588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:15.945 7faaea005588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:15.945 7faaea005588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 05/26/2025 00:44:15.945 7faaea005588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:15.949 7faaf29b9c08 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 05/26/2025 00:44:15.950 7faaea005240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:15.950 7faaea005240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 05/26/2025 00:44:15.950 7faaea005240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:15.961 7faaf29b9c08 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 05/26/2025 00:44:15.961 7faaea0050f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:15.962 7faaea0050f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 05/26/2025 00:44:15.962 7faaea0050f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:15.965 7faaf29b9c08 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 05/26/2025 00:44:15.965 7faaf88ba978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:15.965 7faaf88ba978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 05/26/2025 00:44:15.965 7faaf88ba978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:15.978 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 05/26/2025 00:44:15.978 7faaf29b9c08 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 05/26/2025 00:44:15.978 7faaf29b9c08 [contrail-device-manager] [WARNING]: initializing cassandra cluster 05/26/2025 00:44:15.978 7faaf29b9c08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 05/26/2025 00:44:15.979 7faaef9406d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 05/26/2025 00:44:15.979 7faaef9406d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 05/26/2025 00:44:15.979 7faaec855978 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 05/26/2025 00:44:15.979 7faaec855978 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 05/26/2025 00:44:15.979 7faae9ea1588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:15.980 7faae9ea1588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 05/26/2025 00:44:15.980 7faae9ea1588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:15.981 7faaf29b9c08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 05/26/2025 00:44:15.984 7faaef9406d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 05/26/2025 00:44:15.984 7faaef9406d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 05/26/2025 00:44:15.984 7faaec855978 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 05/26/2025 00:44:16.051 7faaf29b9c08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 05/26/2025 00:44:16.053 7faae9dfe240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:16.053 7faae9dfe240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 05/26/2025 00:44:16.053 7faae9dfe240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:16.060 7faaf29b9c08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 05/26/2025 00:44:16.060 7faaecde08d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:16.060 7faaecde08d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 05/26/2025 00:44:16.060 7faaecde08d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:17.071 7faaf29b9c08 [contrail-device-manager] [WARNING]: Waiting for keyspace config_db_uuid to be created 05/26/2025 00:44:17.080 7faaf29b9c08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 05/26/2025 00:44:17.123 7faaf29b9c08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 05/26/2025 00:44:18.323 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 05/26/2025 00:44:19.357 7faaf29b9c08 [contrail-device-manager] [WARNING]: table 'obj_uuid_table', created 05/26/2025 00:44:19.363 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 05/26/2025 00:44:19.889 7faae9d72c18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:44:19.891 7faae9d72c18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:44:19.891 7faae9d72c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:44:19.891 7faae9d72c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:44:19.891 7faae9d72c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 05/26/2025 00:44:19.891 7faae9d72c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:44:19.891 7faae9d72c18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:44:19.891 7faae9d72c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:44:19.892 7faae9d72c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:19.892 7faae9d72c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 05/26/2025 00:44:19.892 7faae9d72c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:19.892 7faae9d72c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:19.892 7faae9d72c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:44:19.892 7faae9d72c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:21.222 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 05/26/2025 00:44:22.320 7faaf29b9c08 [contrail-device-manager] [WARNING]: table 'obj_uuid_table' fixed with properties {'gc_grace_seconds': 864000} 05/26/2025 00:44:22.326 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 05/26/2025 00:44:23.893 7faae9d72c18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:44:23.894 7faae9d72c18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:44:23.894 7faae9d72c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:44:23.894 7faae9d72c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:44:23.894 7faae9d72c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 05/26/2025 00:44:23.894 7faae9d72c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:44:23.895 7faae9d72c18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:44:23.895 7faae9d72c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:44:23.895 7faae9d72c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:23.895 7faae9d72c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 05/26/2025 00:44:23.895 7faae9d72c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:23.895 7faae9d72c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:23.895 7faae9d72c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:44:23.895 7faae9d72c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:23.934 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 05/26/2025 00:44:25.310 7faaf29b9c08 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table', created 05/26/2025 00:44:25.316 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 05/26/2025 00:44:27.297 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 05/26/2025 00:44:27.894 7faae9deac18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:44:27.896 7faae9deac18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:44:27.896 7faae9deac18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:44:27.896 7faae9deac18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:44:27.896 7faae9deac18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 05/26/2025 00:44:27.896 7faae9deac18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:44:27.896 7faae9deac18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:44:27.897 7faae9deac18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:44:27.897 7faae9deac18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:27.897 7faae9deac18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 05/26/2025 00:44:27.897 7faae9deac18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:27.897 7faae9deac18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:27.897 7faae9deac18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:44:27.897 7faae9deac18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:28.202 7faaf29b9c08 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table' fixed with properties {'gc_grace_seconds': 864000} 05/26/2025 00:44:28.208 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 05/26/2025 00:44:30.055 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 05/26/2025 00:44:30.058 7faaf29b9c08 [contrail-device-manager] [WARNING]: table 'obj_shared_table', created 05/26/2025 00:44:30.063 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 05/26/2025 00:44:31.896 7faae951cb70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:44:31.897 7faae951cb70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:44:31.897 7faae951cb70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:44:31.897 7faae951cb70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:44:31.898 7faae951cb70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 05/26/2025 00:44:31.898 7faae951cb70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:44:31.898 7faae951cb70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:44:31.898 7faae951cb70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:44:31.898 7faae951cb70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:31.898 7faae951cb70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 05/26/2025 00:44:31.898 7faae951cb70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:31.898 7faae951cb70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:31.898 7faae951cb70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:44:31.898 7faae951cb70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:32.204 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 05/26/2025 00:44:33.328 7faaf29b9c08 [contrail-device-manager] [WARNING]: table 'obj_shared_table' fixed with properties {'gc_grace_seconds': 864000} 05/26/2025 00:44:33.334 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 05/26/2025 00:44:33.334 7faae9520390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:33.335 7faae9520390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:44:33.335 7faae9520390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:33.361 7faaf29b9c08 [contrail-device-manager] [ERROR]: Error in db walk read 'type' 05/26/2025 00:44:33.362 7faae952a198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:33.362 7faae952a198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Zookeeper:Zookeeper[] connection down >>, ] >> 05/26/2025 00:44:33.362 7faae952a198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:33.372 7faae9523a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:33.373 7faae9523a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:44:33.373 7faae9523a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:33.373 7faaf29b9c08 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 05/26/2025 00:44:33.373 7faaf29b9c08 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 05/26/2025 00:44:33.374 7faaef9406d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 05/26/2025 00:44:33.375 7faaf29b9c08 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3335-2. 05/26/2025 00:44:33.381 7faaef9406d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 05/26/2025 00:44:33.397 7faaef9406d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3335-2.'] 05/26/2025 00:44:33.408 7faaef9406d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 05/26/2025 00:44:33.412 7faaef9406d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 05/26/2025 00:44:33.424 7faaef9406d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3335-2.'] 05/26/2025 00:44:33.885 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Fetching all containers 05/26/2025 00:44:33.925 7faaf29b9c08 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 05/26/2025 00:44:34.110 7faaf29b9c08 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3335-2..config_queue 05/26/2025 00:44:34.112 7faaf29b9c08 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3335-2..tftp_queue 05/26/2025 00:44:34.435 7faaef9406d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 05/26/2025 00:44:34.440 7faaef9406d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 05/26/2025 00:44:34.473 7faaef9406d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3335-2.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3335-2..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3335-2..tftp_queue'] 05/26/2025 00:44:34.612 7faaf29b9c08 [contrail-device-manager] [WARNING]: Process 138 prepared to run in Full mode... 05/26/2025 00:44:34.613 7faaf29b9c08 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 05/26/2025 00:44:34.613 7faaf29b9c08 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 05/26/2025 00:44:34.615 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page 05/26/2025 00:44:34.615 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 05/26/2025 00:44:34.616 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page 05/26/2025 00:44:34.616 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 05/26/2025 00:44:34.617 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 05/26/2025 00:44:34.617 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 05/26/2025 00:44:34.620 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page 05/26/2025 00:44:34.620 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 05/26/2025 00:44:34.620 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 05/26/2025 00:44:34.620 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 05/26/2025 00:44:34.620 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 05/26/2025 00:44:34.621 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 05/26/2025 00:44:34.621 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 05/26/2025 00:44:34.621 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 05/26/2025 00:44:34.621 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 05/26/2025 00:44:34.621 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 05/26/2025 00:44:34.621 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 05/26/2025 00:44:34.621 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 05/26/2025 00:44:34.621 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 05/26/2025 00:44:34.622 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 05/26/2025 00:44:34.622 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 05/26/2025 00:44:34.622 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 05/26/2025 00:44:34.622 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 05/26/2025 00:44:34.623 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 05/26/2025 00:44:34.623 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 05/26/2025 00:44:34.623 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 05/26/2025 00:44:34.623 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page 05/26/2025 00:44:34.623 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 05/26/2025 00:44:34.623 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 05/26/2025 00:44:34.623 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 05/26/2025 00:44:34.624 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 05/26/2025 00:44:34.624 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 05/26/2025 00:44:34.624 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page 05/26/2025 00:44:34.624 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 05/26/2025 00:44:34.625 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 05/26/2025 00:44:34.625 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 05/26/2025 00:44:34.626 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 05/26/2025 00:44:34.626 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 05/26/2025 00:44:34.626 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 05/26/2025 00:44:34.626 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 05/26/2025 00:44:34.626 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 05/26/2025 00:44:34.626 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 05/26/2025 00:44:34.627 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 05/26/2025 00:44:34.627 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 05/26/2025 00:44:34.627 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 05/26/2025 00:44:34.627 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 05/26/2025 00:44:34.627 7faaf29b9c08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 05/26/2025 00:44:34.629 7faaf29b9c08 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 05/26/2025 00:44:34.644 7faae94d62e8 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 05/26/2025 00:44:34.646 7faae94d62e8 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 05/26/2025 00:44:34.710 7faae8c4e4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:34.710 7faae8c4e4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 05/26/2025 00:44:34.710 7faae8c4e4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:34.724 7faae8b98f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:34.724 7faae8b98f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 05/26/2025 00:44:34.724 7faae8b98f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:35.677 7faaef9406d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 05/26/2025 00:44:35.683 7faaef9406d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 05/26/2025 00:44:35.700 7faaef9406d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3335-2.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3335-2..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3335-2..tftp_queue', 'device_manager_ztp.ztp_queue'] 05/26/2025 00:44:35.898 7faae8ba28d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:44:35.899 7faae8ba28d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:44:35.900 7faae8ba28d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:44:35.900 7faae8ba28d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:44:35.900 7faae8ba28d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 05/26/2025 00:44:35.900 7faae8ba28d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:44:35.900 7faae8ba28d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:44:35.901 7faae8ba28d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:44:35.901 7faae8ba28d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:35.901 7faae8ba28d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 05/26/2025 00:44:35.901 7faae8ba28d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:35.901 7faae8ba28d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:35.901 7faae8ba28d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:44:35.901 7faae8ba28d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:37.846 7faae9472198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:37.847 7faae9472198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:44:37.847 7faae9472198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:39.900 7faae8ab88d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:44:39.901 7faae8ab88d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:44:39.902 7faae8ab88d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:44:39.902 7faae8ab88d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:44:39.902 7faae8ab88d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 05/26/2025 00:44:39.902 7faae8ab88d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:44:39.902 7faae8ab88d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:44:39.902 7faae8ab88d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:44:39.902 7faae8ab88d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:39.902 7faae8ab88d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:44:39.903 7faae8ab88d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:39.903 7faae8ab88d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:39.903 7faae8ab88d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:44:39.903 7faae8ab88d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:43.903 7faae8a598d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:44:43.906 7faae8a598d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:44:43.906 7faae8a598d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:44:43.906 7faae8a598d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:44:43.906 7faae8a598d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 05/26/2025 00:44:43.906 7faae8a598d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:44:43.906 7faae8a598d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:44:43.906 7faae8a598d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:44:43.906 7faae8a598d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:43.907 7faae8a598d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:44:43.907 7faae8a598d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:43.907 7faae8a598d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:43.907 7faae8a598d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:44:43.907 7faae8a598d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:47.906 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:44:47.907 7faae9ed46d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:44:47.907 7faae9ed46d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:44:47.907 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:44:47.907 7faae9ed46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 05/26/2025 00:44:47.907 7faae9ed46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:44:47.907 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:44:47.908 7faae9ed46d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:44:47.908 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:47.908 7faae9ed46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:44:47.908 7faae9ed46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:47.908 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:47.908 7faae9ed46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:44:47.909 7faae9ed46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:51.908 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:44:51.909 7faae9ed46d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:44:51.910 7faae9ed46d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:44:51.910 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:44:51.910 7faae9ed46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 05/26/2025 00:44:51.910 7faae9ed46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:44:51.910 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:44:51.911 7faae9ed46d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:44:51.911 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:51.911 7faae9ed46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:44:51.911 7faae9ed46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:51.911 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:51.911 7faae9ed46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:44:51.911 7faae9ed46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:55.910 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:44:55.912 7faae9ed46d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:44:55.912 7faae9ed46d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:44:55.912 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:44:55.912 7faae9ed46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 05/26/2025 00:44:55.912 7faae9ed46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:44:55.912 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:44:55.913 7faae9ed46d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:44:55.913 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:55.913 7faae9ed46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:44:55.913 7faae9ed46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:55.913 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:55.913 7faae9ed46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:44:55.913 7faae9ed46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:59.913 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:44:59.913 7faae9ed46d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:44:59.914 7faae9ed46d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:44:59.914 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:44:59.914 7faae9ed46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 05/26/2025 00:44:59.914 7faae9ed46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:44:59.914 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:44:59.914 7faae9ed46d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:44:59.915 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:59.915 7faae9ed46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:44:59.915 7faae9ed46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:59.915 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:44:59.915 7faae9ed46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:44:59.915 7faae9ed46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:03.915 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:45:03.917 7faae9ed46d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:45:03.917 7faae9ed46d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:45:03.917 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:03.917 7faae9ed46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 05/26/2025 00:45:03.917 7faae9ed46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:03.917 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:45:03.918 7faae9ed46d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:45:03.918 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:03.918 7faae9ed46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:45:03.918 7faae9ed46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:03.918 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:03.918 7faae9ed46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:45:03.918 7faae9ed46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:07.918 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:45:07.920 7faae9ed46d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:45:07.920 7faae9ed46d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:45:07.920 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:07.920 7faae9ed46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 05/26/2025 00:45:07.920 7faae9ed46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:07.920 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:45:07.920 7faae9ed46d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:45:07.920 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:07.921 7faae9ed46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:45:07.921 7faae9ed46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:07.921 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:07.921 7faae9ed46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:45:07.921 7faae9ed46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:11.921 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:45:11.922 7faae9ed46d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:45:11.922 7faae9ed46d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:45:11.922 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:11.922 7faae9ed46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 05/26/2025 00:45:11.922 7faae9ed46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:11.922 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:45:11.923 7faae9ed46d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:45:11.923 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:11.923 7faae9ed46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:45:11.923 7faae9ed46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:11.923 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:11.923 7faae9ed46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:45:11.923 7faae9ed46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:15.923 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:45:15.925 7faae9ed46d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:45:15.925 7faae9ed46d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:45:15.925 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:15.925 7faae9ed46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 05/26/2025 00:45:15.925 7faae9ed46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:15.925 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:45:15.925 7faae9ed46d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:45:15.925 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:15.926 7faae9ed46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:45:15.926 7faae9ed46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:15.926 7faae9ed46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:15.926 7faae9ed46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:45:15.926 7faae9ed46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:19.925 7faae9d98048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:45:19.927 7faae9d98048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:45:19.927 7faae9d98048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:45:19.928 7faae9d98048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:19.928 7faae9d98048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 05/26/2025 00:45:19.928 7faae9d98048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:19.928 7faae9d98048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:45:19.928 7faae9d98048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:45:19.928 7faae9d98048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:19.929 7faae9d98048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:45:19.929 7faae9d98048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:19.929 7faae9d98048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:19.929 7faae9d98048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:45:19.929 7faae9d98048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:23.928 7faae94850f0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:45:23.929 7faae94850f0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:45:23.929 7faae94850f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:45:23.929 7faae94850f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:23.929 7faae94850f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 05/26/2025 00:45:23.929 7faae94850f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:23.929 7faae94850f0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:45:23.930 7faae94850f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:45:23.930 7faae94850f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:23.930 7faae94850f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:45:23.930 7faae94850f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:23.930 7faae94850f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:23.930 7faae94850f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:45:23.930 7faae94850f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:27.930 7faae9e57b70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:45:27.931 7faae9e57b70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:45:27.931 7faae9e57b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:45:27.931 7faae9e57b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:27.932 7faae9e57b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 05/26/2025 00:45:27.932 7faae9e57b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:27.932 7faae9e57b70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:45:27.932 7faae9e57b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:45:27.932 7faae9e57b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:27.932 7faae9e57b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:45:27.932 7faae9e57b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:27.932 7faae9e57b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:27.932 7faae9e57b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:45:27.932 7faae9e57b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:31.932 7faae8a5ef60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:45:31.933 7faae8a5ef60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:45:31.934 7faae8a5ef60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:45:31.934 7faae8a5ef60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:31.934 7faae8a5ef60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 05/26/2025 00:45:31.934 7faae8a5ef60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:31.934 7faae8a5ef60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:45:31.934 7faae8a5ef60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:45:31.934 7faae8a5ef60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:31.934 7faae8a5ef60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:45:31.934 7faae8a5ef60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:31.934 7faae8a5ef60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:31.935 7faae8a5ef60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:45:31.935 7faae8a5ef60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:35.934 7faaefb67438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:45:35.935 7faaefb67438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:45:35.935 7faaefb67438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:45:35.935 7faaefb67438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:35.936 7faaefb67438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 05/26/2025 00:45:35.936 7faaefb67438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:35.936 7faaefb67438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:45:35.936 7faaefb67438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:45:35.936 7faaefb67438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:35.936 7faaefb67438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:45:35.936 7faaefb67438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:35.936 7faaefb67438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:35.936 7faaefb67438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:45:35.936 7faaefb67438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:39.936 7faae95234e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:45:39.937 7faae95234e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:45:39.937 7faae95234e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:45:39.937 7faae95234e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:39.938 7faae95234e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 05/26/2025 00:45:39.938 7faae95234e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:39.938 7faae95234e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:45:39.938 7faae95234e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:45:39.938 7faae95234e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:39.939 7faae95234e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:45:39.939 7faae95234e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:39.939 7faae95234e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:39.939 7faae95234e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:45:39.939 7faae95234e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:43.938 7faae946ba20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:45:43.939 7faae946ba20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:45:43.939 7faae946ba20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:45:43.939 7faae946ba20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:43.939 7faae946ba20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 05/26/2025 00:45:43.939 7faae946ba20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:43.940 7faae946ba20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:45:43.940 7faae946ba20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:45:43.940 7faae946ba20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:43.940 7faae946ba20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:45:43.940 7faae946ba20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:43.940 7faae946ba20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:43.940 7faae946ba20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:45:43.940 7faae946ba20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:47.939 7faae8c4ed68 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:45:47.940 7faae8c4ed68 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:45:47.940 7faae8c4ed68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:45:47.940 7faae8c4ed68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:47.941 7faae8c4ed68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 05/26/2025 00:45:47.941 7faae8c4ed68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:47.941 7faae8c4ed68 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:45:47.941 7faae8c4ed68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:45:47.941 7faae8c4ed68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:47.941 7faae8c4ed68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:45:47.941 7faae8c4ed68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:47.941 7faae8c4ed68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:47.941 7faae8c4ed68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:45:47.941 7faae8c4ed68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:51.940 7faae951c978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:45:51.941 7faae951c978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:45:51.942 7faae951c978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:45:51.942 7faae951c978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:51.942 7faae951c978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 05/26/2025 00:45:51.942 7faae951c978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:51.942 7faae951c978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:45:51.942 7faae951c978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:45:51.942 7faae951c978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:51.943 7faae951c978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:45:51.943 7faae951c978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:51.943 7faae951c978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:51.943 7faae951c978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:45:51.943 7faae951c978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:55.077 7faaf29b9c08 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 05/26/2025 00:45:55.077 7faaf29b9c08 [contrail-device-manager] [WARNING]: initializing cassandra cluster 05/26/2025 00:45:55.077 7faaf29b9c08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 05/26/2025 00:45:55.077 7faaf0d45d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:55.078 7faaf0d45d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 05/26/2025 00:45:55.078 7faaf0d45d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:55.078 7faaf29b9c08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 05/26/2025 00:45:55.102 7faaf29b9c08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 05/26/2025 00:45:55.103 7faae8c64828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:55.104 7faae8c64828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Initializing description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 05/26/2025 00:45:55.104 7faae8c64828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:55.110 7faaf29b9c08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 05/26/2025 00:45:55.110 7faae8c64630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:55.110 7faae8c64630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 05/26/2025 00:45:55.110 7faae8c64630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:55.122 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 05/26/2025 00:45:55.942 7faae8aaaa20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:45:55.944 7faae8aaaa20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:45:55.944 7faae8aaaa20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:45:55.944 7faae8aaaa20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:55.945 7faae8aaaa20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 05/26/2025 00:45:55.945 7faae8aaaa20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:55.945 7faae8aaaa20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:45:55.945 7faae8aaaa20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:45:55.945 7faae8aaaa20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:55.945 7faae8aaaa20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 05/26/2025 00:45:55.945 7faae8aaaa20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:55.945 7faae8aaaa20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:55.946 7faae8aaaa20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:45:55.946 7faae8aaaa20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:57.012 7faaf29b9c08 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace', created 05/26/2025 00:45:57.068 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 05/26/2025 00:45:57.080 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 05/26/2025 00:45:58.317 7faaf29b9c08 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None} 05/26/2025 00:45:58.322 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 05/26/2025 00:45:58.322 7faaf29b9c08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 05/26/2025 00:45:58.406 7faaf29b9c08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 05/26/2025 00:45:58.420 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 05/26/2025 00:45:59.944 7faae8c1cd68 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:45:59.945 7faae8c1cd68 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:45:59.945 7faae8c1cd68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:45:59.945 7faae8c1cd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:59.946 7faae8c1cd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 05/26/2025 00:45:59.946 7faae8c1cd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:45:59.946 7faae8c1cd68 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:45:59.946 7faae8c1cd68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:45:59.946 7faae8c1cd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:59.946 7faae8c1cd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 05/26/2025 00:45:59.946 7faae8c1cd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:59.946 7faae8c1cd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:45:59.946 7faae8c1cd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:45:59.946 7faae8c1cd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:00.107 7faaf29b9c08 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table', created 05/26/2025 00:46:00.116 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 05/26/2025 00:46:00.127 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 05/26/2025 00:46:01.470 7faaf29b9c08 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table' fixed with properties {'gc_grace_seconds': 864000} 05/26/2025 00:46:01.474 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 05/26/2025 00:46:01.481 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 05/26/2025 00:46:02.456 7faaf29b9c08 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table', created 05/26/2025 00:46:02.460 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 05/26/2025 00:46:02.468 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 05/26/2025 00:46:02.974 7faaf29b9c08 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table' fixed with properties {'gc_grace_seconds': 864000} 05/26/2025 00:46:02.978 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 05/26/2025 00:46:02.986 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 05/26/2025 00:46:03.946 7faae8a36c18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:46:03.947 7faae8a36c18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:46:03.947 7faae8a36c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:46:03.947 7faae8a36c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:46:03.948 7faae8a36c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 05/26/2025 00:46:03.948 7faae8a36c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:46:03.948 7faae8a36c18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:46:03.948 7faae8a36c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:46:03.948 7faae8a36c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:03.948 7faae8a36c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 05/26/2025 00:46:03.948 7faae8a36c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:03.948 7faae8a36c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:03.948 7faae8a36c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:46:03.948 7faae8a36c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:03.952 7faaf29b9c08 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table', created 05/26/2025 00:46:03.956 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 05/26/2025 00:46:03.962 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 05/26/2025 00:46:04.468 7faaf29b9c08 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table' fixed with properties {'gc_grace_seconds': 864000} 05/26/2025 00:46:04.473 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 05/26/2025 00:46:04.479 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 05/26/2025 00:46:05.211 7faaf29b9c08 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table', created 05/26/2025 00:46:05.217 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 05/26/2025 00:46:05.223 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 05/26/2025 00:46:06.194 7faaf29b9c08 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table' fixed with properties {'gc_grace_seconds': 864000} 05/26/2025 00:46:06.200 7faaf29b9c08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 05/26/2025 00:46:06.201 7faae8add588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:06.201 7faae8add588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:46:06.201 7faae8add588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:06.282 7faaf29b9c08 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.19:5673//', 'pyamqp://guest:guest@10.0.0.41:5673//', 'pyamqp://guest:guest@10.0.0.61:5673//'] 05/26/2025 00:46:06.282 7faaf29b9c08 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 05/26/2025 00:46:06.283 7faae8a744e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:06.283 7faae8a744e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Initializing description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 05/26/2025 00:46:06.283 7faae8a744e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:06.283 7faae8a744e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:06.283 7faae8a744e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 05/26/2025 00:46:06.283 7faae8a744e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:06.287 7faaf29b9c08 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 05/26/2025 00:46:06.287 7faae8c1cd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:06.287 7faae8c1cd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:46:06.287 7faae8c1cd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:06.328 7faaf29b9c08 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 05/26/2025 00:46:06.329 7faae8add240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:06.330 7faae8add240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 05/26/2025 00:46:06.330 7faae8add240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:06.334 7faaf29b9c08 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 05/26/2025 00:46:06.334 7faae8c1cd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:06.334 7faae8c1cd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:46:06.334 7faae8c1cd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:06.335 7faae8c276d8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 05/26/2025 00:46:06.336 7faae8a36048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:06.336 7faae8a36048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 05/26/2025 00:46:06.336 7faae8a36048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:06.342 7faae8c276d8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 05/26/2025 00:46:06.342 7faae8aa5e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:06.342 7faae8aa5e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:46:06.342 7faae8aa5e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:07.947 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:46:07.949 7faae8c18828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:46:07.949 7faae8c18828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:46:07.949 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:46:07.949 7faae8c18828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 05/26/2025 00:46:07.949 7faae8c18828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:46:07.949 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:46:07.950 7faae8c18828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:46:07.950 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:07.950 7faae8c18828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:46:07.950 7faae8c18828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:07.950 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:07.950 7faae8c18828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:46:07.950 7faae8c18828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:11.950 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:46:11.951 7faae8c18828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:46:11.951 7faae8c18828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:46:11.951 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:46:11.951 7faae8c18828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 05/26/2025 00:46:11.951 7faae8c18828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:46:11.952 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:46:11.952 7faae8c18828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:46:11.952 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:11.952 7faae8c18828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:46:11.952 7faae8c18828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:11.952 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:11.952 7faae8c18828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:46:11.952 7faae8c18828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:15.951 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:46:15.953 7faae8c18828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:46:15.953 7faae8c18828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:46:15.953 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:46:15.953 7faae8c18828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 05/26/2025 00:46:15.953 7faae8c18828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:46:15.953 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:46:15.953 7faae8c18828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:46:15.953 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:15.954 7faae8c18828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:46:15.954 7faae8c18828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:15.954 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:15.954 7faae8c18828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:46:15.954 7faae8c18828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:19.953 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:46:19.955 7faae8c18828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:46:19.955 7faae8c18828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:46:19.955 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:46:19.955 7faae8c18828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 05/26/2025 00:46:19.955 7faae8c18828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:46:19.955 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:46:19.956 7faae8c18828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:46:19.956 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:19.956 7faae8c18828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:46:19.956 7faae8c18828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:19.956 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:19.956 7faae8c18828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:46:19.956 7faae8c18828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:23.955 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:46:23.956 7faae8c18828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:46:23.956 7faae8c18828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:46:23.956 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:46:23.956 7faae8c18828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 05/26/2025 00:46:23.956 7faae8c18828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:46:23.956 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:46:23.957 7faae8c18828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:46:23.957 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:23.957 7faae8c18828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:46:23.957 7faae8c18828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:23.957 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:23.957 7faae8c18828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:46:23.957 7faae8c18828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:27.956 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:46:27.958 7faae8c18828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 05/26/2025 00:46:27.958 7faae8c18828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:46:27.958 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:46:27.958 7faae8c18828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 05/26/2025 00:46:27.958 7faae8c18828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:46:27.958 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 05/26/2025 00:46:27.959 7faae8c18828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 05/26/2025 00:46:27.959 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:27.959 7faae8c18828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 05/26/2025 00:46:27.959 7faae8c18828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:27.959 7faae8c18828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:27.959 7faae8c18828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1748220255889771 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.61:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 05/26/2025 00:46:27.959 7faae8c18828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/26/2025 00:46:29.389 7faae8c1cd68 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3335-1.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.19', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3335-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3335-1.'], 'id_perms': {'created': '2025-05-26T00:46:29.378324', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-05-26T00:46:29.378324', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11669344748968103690, 'uuid_mslong': 17910634230755575392}}, 'parent_type': 'global-system-config', 'parent_uuid': 'be5ad7d3-1178-4abe-bce5-c6fed561d326', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'f88f5af0-3223-4660-a1f1-d738ebd04f0a'}, 'oper': 'CREATE', 'request-id': 'req-f2294937-8beb-454e-8093-ce53dfe19092', 'type': 'analytics_node', 'uuid': 'f88f5af0-3223-4660-a1f1-d738ebd04f0a'} 05/26/2025 00:46:29.715 7faae8c1cd68 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3335-2.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.41', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3335-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3335-2.'], 'id_perms': {'created': '2025-05-26T00:46:29.707794', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-05-26T00:46:29.707794', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11378416737474160644, 'uuid_mslong': 11609184381964927092}}, 'parent_type': 'global-system-config', 'parent_uuid': 'be5ad7d3-1178-4abe-bce5-c6fed561d326', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'a11c1baf-f308-4074-9de8-41bcbf81a804'}, 'oper': 'CREATE', 'request-id': 'req-ff3b1732-8472-4ec3-a272-22a7f335e92c', 'type': 'analytics_node', 'uuid': 'a11c1baf-f308-4074-9de8-41bcbf81a804'} 05/26/2025 00:46:29.781 7faae8c1cd68 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3335-3.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.61', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3335-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3335-3.'], 'id_perms': {'created': '2025-05-26T00:46:29.770684', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-05-26T00:46:29.770684', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9451594112470072585, 'uuid_mslong': 11008946774828992063}}, 'parent_type': 'global-system-config', 'parent_uuid': 'be5ad7d3-1178-4abe-bce5-c6fed561d326', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '98c7a2c3-88dd-463f-832a-cec4c2f3bd09'}, 'oper': 'CREATE', 'request-id': 'req-1774bbcb-50b5-46d4-b530-d7c93291bcb2', 'type': 'analytics_node', 'uuid': '98c7a2c3-88dd-463f-832a-cec4c2f3bd09'} 05/26/2025 00:46:31.958 7faae8950a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/26/2025 00:46:31.959 7faae8950a20 [contrail-device-manager] [INFO]: Session Event: TCP Connected 05/26/2025 00:46:31.959 7faae8950a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/26/2025 00:46:31.960 7faae8950a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:46:31.960 7faae8950a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3335-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 05/26/2025 00:46:31.961 7faae8950a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/26/2025 00:46:31.961 7faae8950a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect] 05/26/2025 00:46:31.961 7faae8950a20 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19 05/26/2025 00:46:31.961 7faae8950a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 05/26/2025 00:46:31.961 7faae8950a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 05/26/2025 00:46:31.961 7faae8950a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 05/26/2025 00:46:31.962 7faae894c438 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 05/26/2025 00:46:31.964 7faae894c438 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 05/26/2025 00:46:39.900 7faae894c630 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001016 05/26/2025 00:46:39.901 7faae894c630 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 05/26/2025 00:46:39.901 7faae94cca20 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 05/26/2025 00:46:39.901 7faae94cca20 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 0 05/26/2025 00:46:39.901 7faae94cca20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 05/26/2025 00:46:39.901 7faae94cca20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 05/26/2025 00:46:39.901 7faae94cca20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 05/26/2025 00:46:39.902 7faae894e0f0 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 05/26/2025 00:46:39.903 7faae894e0f0 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 05/26/2025 00:46:44.617 7faae8c1cd68 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3335-2.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.41', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3335-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3335-2.'], 'id_perms': {'created': '2025-05-26T00:46:44.601540', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-05-26T00:46:44.601540', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11392439510888742954, 'uuid_mslong': 6631730637834634478}}, 'parent_type': 'global-system-config', 'parent_uuid': 'be5ad7d3-1178-4abe-bce5-c6fed561d326', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '5c08a3e0-f235-44ee-9e1a-13601ba6002a'}, 'oper': 'CREATE', 'request-id': 'req-0bd487af-0f6c-4814-9900-4ed7a44d179c', 'type': 'analytics_alarm_node', 'uuid': '5c08a3e0-f235-44ee-9e1a-13601ba6002a'} 05/26/2025 00:46:46.238 7faae8c1cd68 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3335-3.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.61', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3335-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3335-3.'], 'id_perms': {'created': '2025-05-26T00:46:46.215715', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-05-26T00:46:46.215715', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9405677528990650479, 'uuid_mslong': 15792557173547420287}}, 'parent_type': 'global-system-config', 'parent_uuid': 'be5ad7d3-1178-4abe-bce5-c6fed561d326', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'db2a6f13-a857-4a7f-8287-ade1135c906f'}, 'oper': 'CREATE', 'request-id': 'req-f21c3298-e794-40a3-ae6f-6f199f5e7cdf', 'type': 'analytics_alarm_node', 'uuid': 'db2a6f13-a857-4a7f-8287-ade1135c906f'} 05/26/2025 00:46:46.281 7faae8c1cd68 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3335-1.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.19', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3335-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3335-1.'], 'id_perms': {'created': '2025-05-26T00:46:46.265383', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-05-26T00:46:46.265383', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11073789822056512541, 'uuid_mslong': 12472971718646317071}}, 'parent_type': 'global-system-config', 'parent_uuid': 'be5ad7d3-1178-4abe-bce5-c6fed561d326', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'ad18e5b1-8768-400f-99ae-012bf503bc1d'}, 'oper': 'CREATE', 'request-id': 'req-3e493844-6006-46a5-a786-768fb5c5710d', 'type': 'analytics_alarm_node', 'uuid': 'ad18e5b1-8768-400f-99ae-012bf503bc1d'} 05/26/2025 00:46:50.184 7faae894c630 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001296 05/26/2025 00:46:50.185 7faae894c630 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 05/26/2025 00:46:50.185 7faae8aad630 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 05/26/2025 00:46:50.185 7faae8aad630 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 2 05/26/2025 00:46:50.185 7faae8aad630 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 05/26/2025 00:46:50.185 7faae8aad630 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 05/26/2025 00:46:56.096 7faae8c1cd68 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3335-2.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.41', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3335-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3335-2.'], 'id_perms': {'created': '2025-05-26T00:46:56.081607', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-05-26T00:46:56.081607', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10713303818700755812, 'uuid_mslong': 5801532066866217952}}, 'parent_type': 'global-system-config', 'parent_uuid': 'be5ad7d3-1178-4abe-bce5-c6fed561d326', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '50832ead-4e99-43e0-94ad-4d0fc47e9364'}, 'oper': 'CREATE', 'request-id': 'req-7ba4c7ed-155d-4e70-8b22-7fef2b030972', 'type': 'analytics_snmp_node', 'uuid': '50832ead-4e99-43e0-94ad-4d0fc47e9364'} 05/26/2025 00:46:56.180 7faae8c1cd68 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3335-3.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.61', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3335-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3335-3.'], 'id_perms': {'created': '2025-05-26T00:46:56.159666', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-05-26T00:46:56.159666', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11924576475093683733, 'uuid_mslong': 7416259045133273651}}, 'parent_type': 'global-system-config', 'parent_uuid': 'be5ad7d3-1178-4abe-bce5-c6fed561d326', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '66ebd84e-fb72-4a33-a57c-9b1ffef04e15'}, 'oper': 'CREATE', 'request-id': 'req-c4be7497-8c46-46a2-8a61-ffd8fbf6d6df', 'type': 'analytics_snmp_node', 'uuid': '66ebd84e-fb72-4a33-a57c-9b1ffef04e15'} 05/26/2025 00:46:56.198 7faae8c1cd68 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3335-1.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.19', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3335-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3335-1.'], 'id_perms': {'created': '2025-05-26T00:46:56.167020', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-05-26T00:46:56.167020', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11987365680041062837, 'uuid_mslong': 3066254195269060315}}, 'parent_type': 'global-system-config', 'parent_uuid': 'be5ad7d3-1178-4abe-bce5-c6fed561d326', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '2a8d85f1-e410-46db-a65b-ad92ff8e71b5'}, 'oper': 'CREATE', 'request-id': 'req-e8cb7265-a2ad-48fa-b082-1804276a2f10', 'type': 'analytics_snmp_node', 'uuid': '2a8d85f1-e410-46db-a65b-ad92ff8e71b5'} 05/26/2025 00:48:36.460 7faae8c1cd68 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-4d1a0e1f-4c03-485f-b151-5af49dfe1674', 'type': 'global_system_config', 'uuid': 'be5ad7d3-1178-4abe-bce5-c6fed561d326'} 05/26/2025 00:48:36.523 7faae8c1cd68 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-d1870c13-2211-4d39-9661-712cef5efc58', 'type': 'global_system_config', 'uuid': 'be5ad7d3-1178-4abe-bce5-c6fed561d326'} 05/26/2025 00:48:36.596 7faae8c1cd68 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3335-1'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-3335-1', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3335-1'], 'id_perms': {'created': '2025-05-26T00:48:36.586526', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-05-26T00:48:36.586526', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12774880448280109631, 'uuid_mslong': 5962796790867445186}}, 'parent_type': 'global-system-config', 'parent_uuid': 'be5ad7d3-1178-4abe-bce5-c6fed561d326', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '52c01c16-cb7e-45c2-b149-7e155fd1a23f', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.193'}, 'oper': 'CREATE', 'request-id': 'req-311f92bb-457e-4ba4-953c-0684a56b3434', 'type': 'virtual_router', 'uuid': '52c01c16-cb7e-45c2-b149-7e155fd1a23f'} 05/26/2025 00:48:36.656 7faae8c1cd68 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3335-2'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-3335-2', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3335-2'], 'id_perms': {'created': '2025-05-26T00:48:36.648095', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-05-26T00:48:36.648095', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12350364892800903156, 'uuid_mslong': 1020463542237219562}}, 'parent_type': 'global-system-config', 'parent_uuid': 'be5ad7d3-1178-4abe-bce5-c6fed561d326', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '0e296a2f-5d49-4aea-ab65-4f6f59da67f4', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.129'}, 'oper': 'CREATE', 'request-id': 'req-b10b19c9-0a26-4d43-ab8c-b9ebdf55641c', 'type': 'virtual_router', 'uuid': '0e296a2f-5d49-4aea-ab65-4f6f59da67f4'} 05/26/2025 00:48:36.945 7faae8c1cd68 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3335-1', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3335-1', 'vhost0'], 'id_perms': {'created': '2025-05-26T00:48:36.931363', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-05-26T00:48:36.931363', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10043135547604446531, 'uuid_mslong': 11264600474018859880}}, 'parent_type': 'virtual-router', 'parent_uuid': '52c01c16-cb7e-45c2-b149-7e155fd1a23f', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '9c53e672-de97-4b68-8b60-6291da0fad43', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:9c:53:e6:72:de']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': '82708503-f566-4e0e-b4cb-ebbe20964732'}]}, 'oper': 'CREATE', 'request-id': 'req-aa6d91af-7ff1-40e9-9fcc-3261c606e896', 'type': 'virtual_machine_interface', 'uuid': '9c53e672-de97-4b68-8b60-6291da0fad43'} 05/26/2025 00:48:36.988 7faae8c1cd68 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3335-1', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-aa6d91af-7ff1-40e9-9fcc-3261c606e896', 'type': 'virtual_machine_interface', 'uuid': '9c53e672-de97-4b68-8b60-6291da0fad43'} 05/26/2025 00:48:37.020 7faae8c1cd68 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3335-2', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3335-2', 'vhost0'], 'id_perms': {'created': '2025-05-26T00:48:37.001883', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-05-26T00:48:37.001883', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11163773284619950605, 'uuid_mslong': 12968013669036409312}}, 'parent_type': 'virtual-router', 'parent_uuid': '0e296a2f-5d49-4aea-ab65-4f6f59da67f4', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'b3f7a3b9-7953-41e0-9aed-b0a776c49a0d', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:b3:f7:a3:b9:79']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': '82708503-f566-4e0e-b4cb-ebbe20964732'}]}, 'oper': 'CREATE', 'request-id': 'req-89e6a523-0aa4-494d-bed1-4f24bf019a36', 'type': 'virtual_machine_interface', 'uuid': 'b3f7a3b9-7953-41e0-9aed-b0a776c49a0d'} 05/26/2025 00:48:37.064 7faae8c1cd68 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3335-2', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-89e6a523-0aa4-494d-bed1-4f24bf019a36', 'type': 'virtual_machine_interface', 'uuid': 'b3f7a3b9-7953-41e0-9aed-b0a776c49a0d'} 05/26/2025 00:48:37.493 7faae8c1cd68 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-b6596350-7653-4b17-923a-60a6ad34b6cc', 'type': 'access_control_list', 'uuid': 'e5c80a49-54b2-432e-bbcc-f4e25e4e595d'} 05/26/2025 00:49:21.201 7faae8a40b70 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 05/26/2025 00:49:39.709 7faae8a40b70 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 05/26/2025 01:53:17.202 7faae94cca20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map