09/04/2025 05:12:47.672 7f045b1038c8 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/04/2025 05:12:47.672 7f045b1038c8 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/04/2025 05:12:47.672 7f045b1038c8 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/04/2025 05:12:47.672 7f04527432e8 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 09/04/2025 05:12:47.673 7f04527432e8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 09/04/2025 05:12:47.673 7f04527432e8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/04/2025 05:12:47.673 7f04527432e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/04/2025 05:12:47.673 7f04527432e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:47.674 7f04527432e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/04/2025 05:12:47.674 7f04527432e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:47.674 7f04527432e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:47.674 7f04527432e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 start_time = 1756962767673643 collector_name = collector_ip = collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/04/2025 05:12:47.674 7f04527432e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:47.720 7f045b1038c8 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.254:5673//', 'pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/04/2025 05:12:47.730 7f045b1038c8 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 09/04/2025 05:12:47.730 7f0452650588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:47.731 7f0452650588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 09/04/2025 05:12:47.731 7f0452650588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:47.731 7f0452650588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:47.731 7f0452650588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 09/04/2025 05:12:47.731 7f0452650588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:47.735 7f045b1038c8 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/04/2025 05:12:47.736 7f0452650240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:47.736 7f0452650240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/04/2025 05:12:47.736 7f0452650240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:47.749 7f045b1038c8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 09/04/2025 05:12:47.750 7f04526500f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:47.750 7f04526500f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 09/04/2025 05:12:47.750 7f04526500f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:47.754 7f045b1038c8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/04/2025 05:12:47.754 7f0460f45978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:47.755 7f0460f45978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/04/2025 05:12:47.755 7f0460f45978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:47.868 7f045b1038c8 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 09/04/2025 05:12:47.869 7f045b1038c8 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 09/04/2025 05:12:47.869 7f045b1038c8 [contrail-device-manager] [WARNING]: initializing cassandra cluster 09/04/2025 05:12:47.869 7f045b1038c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/04/2025 05:12:47.869 7f0457fca6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 09/04/2025 05:12:47.870 7f0457fca6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 09/04/2025 05:12:47.870 7f0454ed5978 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 09/04/2025 05:12:47.870 7f0454ed5978 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 09/04/2025 05:12:47.870 7f045252e588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:47.870 7f045252e588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 09/04/2025 05:12:47.870 7f045252e588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:47.872 7f045b1038c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/04/2025 05:12:47.874 7f0457fca6d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 09/04/2025 05:12:47.874 7f0457fca6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 09/04/2025 05:12:47.876 7f0454ed5978 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 09/04/2025 05:12:47.932 7f045b1038c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/04/2025 05:12:47.935 7f04524cd828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:47.935 7f04524cd828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/04/2025 05:12:47.936 7f04524cd828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:47.944 7f045b1038c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 09/04/2025 05:12:47.944 7f0452455c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:47.945 7f0452455c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 09/04/2025 05:12:47.945 7f0452455c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:48.955 7f045b1038c8 [contrail-device-manager] [WARNING]: Waiting for keyspace config_db_uuid to be created 09/04/2025 05:12:48.965 7f045b1038c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 09/04/2025 05:12:49.016 7f045b1038c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 09/04/2025 05:12:50.793 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/04/2025 05:12:51.673 7f04524f78d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:12:51.675 7f04524f78d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:12:51.675 7f04524f78d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:12:51.675 7f04524f78d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:12:51.675 7f04524f78d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/04/2025 05:12:51.675 7f04524f78d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:12:51.675 7f04524f78d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:12:51.676 7f04524f78d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:12:51.676 7f04524f78d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:51.676 7f04524f78d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/04/2025 05:12:51.676 7f04524f78d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:51.676 7f04524f78d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:51.676 7f04524f78d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:12:51.676 7f04524f78d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:51.745 7f045b1038c8 [contrail-device-manager] [WARNING]: table 'obj_uuid_table', created 09/04/2025 05:12:51.749 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/04/2025 05:12:52.842 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/04/2025 05:12:53.967 7f045b1038c8 [contrail-device-manager] [WARNING]: table 'obj_uuid_table' fixed with properties {'gc_grace_seconds': 864000} 09/04/2025 05:12:53.972 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/04/2025 05:12:55.675 7f04524b8390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:12:55.676 7f04524b8390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:12:55.677 7f04524b8390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:12:55.677 7f04524b8390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:12:55.677 7f04524b8390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/04/2025 05:12:55.677 7f04524b8390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:12:55.677 7f04524b8390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:12:55.677 7f04524b8390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:12:55.677 7f04524b8390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:55.678 7f04524b8390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/04/2025 05:12:55.678 7f04524b8390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:55.678 7f04524b8390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:55.678 7f04524b8390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:12:55.678 7f04524b8390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:55.884 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/04/2025 05:12:56.758 7f045b1038c8 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table', created 09/04/2025 05:12:56.767 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/04/2025 05:12:59.677 7f0452421e10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:12:59.679 7f0452421e10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:12:59.679 7f0452421e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:12:59.679 7f0452421e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:12:59.679 7f0452421e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/04/2025 05:12:59.679 7f0452421e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:12:59.679 7f0452421e10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:12:59.679 7f0452421e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:12:59.679 7f0452421e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:59.680 7f0452421e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/04/2025 05:12:59.680 7f0452421e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:59.680 7f0452421e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:59.680 7f0452421e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:12:59.680 7f0452421e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:12:59.705 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/04/2025 05:13:00.814 7f045b1038c8 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table' fixed with properties {'gc_grace_seconds': 864000} 09/04/2025 05:13:00.818 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/04/2025 05:13:02.737 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/04/2025 05:13:02.741 7f045b1038c8 [contrail-device-manager] [WARNING]: table 'obj_shared_table', created 09/04/2025 05:13:02.747 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/04/2025 05:13:03.680 7f0452415d68 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:13:03.681 7f0452415d68 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:13:03.682 7f0452415d68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:13:03.682 7f0452415d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:03.682 7f0452415d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/04/2025 05:13:03.682 7f0452415d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:03.682 7f0452415d68 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:13:03.682 7f0452415d68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:13:03.682 7f0452415d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:03.682 7f0452415d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/04/2025 05:13:03.682 7f0452415d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:03.683 7f0452415d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:03.683 7f0452415d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:13:03.683 7f0452415d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:04.904 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/04/2025 05:13:05.869 7f045b1038c8 [contrail-device-manager] [WARNING]: table 'obj_shared_table' fixed with properties {'gc_grace_seconds': 864000} 09/04/2025 05:13:05.874 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/04/2025 05:13:05.874 7f04524cd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:05.875 7f04524cd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:13:05.875 7f04524cd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:05.909 7f045b1038c8 [contrail-device-manager] [ERROR]: Error in db walk read 'type' 09/04/2025 05:13:05.910 7f045241e438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:05.911 7f045241e438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Zookeeper:Zookeeper[] connection down >>, ] >> 09/04/2025 05:13:05.911 7f045241e438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:05.925 7f0452418e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:05.925 7f0452418e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:13:05.925 7f0452418e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:05.925 7f045b1038c8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 09/04/2025 05:13:05.926 7f045b1038c8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 09/04/2025 05:13:05.926 7f0457fca6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 09/04/2025 05:13:05.927 7f045b1038c8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4013-3. 09/04/2025 05:13:05.933 7f0457fca6d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 09/04/2025 05:13:05.953 7f0457fca6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4013-3.'] 09/04/2025 05:13:05.964 7f0457fca6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 09/04/2025 05:13:05.983 7f0457fca6d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 09/04/2025 05:13:06.008 7f0457fca6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4013-3.'] 09/04/2025 05:13:06.443 7f045b1038c8 [contrail-device-manager] [DEBUG]: Fetching all containers 09/04/2025 05:13:06.503 7f045b1038c8 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 09/04/2025 05:13:06.745 7f045b1038c8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4013-3..config_queue 09/04/2025 05:13:06.746 7f045b1038c8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4013-3..tftp_queue 09/04/2025 05:13:07.021 7f0457fca6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 09/04/2025 05:13:07.036 7f0457fca6d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 09/04/2025 05:13:07.055 7f0457fca6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4013-3.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4013-3..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4013-3..tftp_queue'] 09/04/2025 05:13:07.247 7f045b1038c8 [contrail-device-manager] [WARNING]: Process 139 prepared to run in Full mode... 09/04/2025 05:13:07.247 7f045b1038c8 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 09/04/2025 05:13:07.247 7f045b1038c8 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 09/04/2025 05:13:07.252 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page 09/04/2025 05:13:07.252 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 09/04/2025 05:13:07.253 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page 09/04/2025 05:13:07.253 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 09/04/2025 05:13:07.254 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 09/04/2025 05:13:07.255 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 09/04/2025 05:13:07.260 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page 09/04/2025 05:13:07.260 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 09/04/2025 05:13:07.261 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 09/04/2025 05:13:07.261 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 09/04/2025 05:13:07.261 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 09/04/2025 05:13:07.261 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 09/04/2025 05:13:07.261 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 09/04/2025 05:13:07.261 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 09/04/2025 05:13:07.261 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 09/04/2025 05:13:07.261 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 09/04/2025 05:13:07.262 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 09/04/2025 05:13:07.262 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 09/04/2025 05:13:07.262 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 09/04/2025 05:13:07.264 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 09/04/2025 05:13:07.264 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 09/04/2025 05:13:07.264 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 09/04/2025 05:13:07.264 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 09/04/2025 05:13:07.265 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 09/04/2025 05:13:07.265 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 09/04/2025 05:13:07.265 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 09/04/2025 05:13:07.266 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page 09/04/2025 05:13:07.266 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 09/04/2025 05:13:07.266 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 09/04/2025 05:13:07.266 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 09/04/2025 05:13:07.266 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 09/04/2025 05:13:07.266 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 09/04/2025 05:13:07.268 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page 09/04/2025 05:13:07.268 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 09/04/2025 05:13:07.268 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 09/04/2025 05:13:07.268 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 09/04/2025 05:13:07.268 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 09/04/2025 05:13:07.268 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 09/04/2025 05:13:07.268 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 09/04/2025 05:13:07.269 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 09/04/2025 05:13:07.269 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 09/04/2025 05:13:07.269 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 09/04/2025 05:13:07.269 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 09/04/2025 05:13:07.269 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 09/04/2025 05:13:07.269 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 09/04/2025 05:13:07.269 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 09/04/2025 05:13:07.269 7f045b1038c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 09/04/2025 05:13:07.273 7f045b1038c8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 09/04/2025 05:13:07.294 7f0451bb7390 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 09/04/2025 05:13:07.294 7f0451bb7390 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 09/04/2025 05:13:07.371 7f045132df60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:07.372 7f045132df60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 09/04/2025 05:13:07.372 7f045132df60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:07.395 7f04512194e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:07.395 7f04512194e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 09/04/2025 05:13:07.395 7f04512194e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:07.681 7f0451ba68d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:13:07.683 7f0451ba68d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:13:07.683 7f0451ba68d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:13:07.683 7f0451ba68d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:07.683 7f0451ba68d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 09/04/2025 05:13:07.683 7f0451ba68d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:07.683 7f0451ba68d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:13:07.683 7f0451ba68d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:13:07.683 7f0451ba68d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:07.684 7f0451ba68d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 09/04/2025 05:13:07.684 7f0451ba68d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:07.684 7f0451ba68d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:07.684 7f0451ba68d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:13:07.684 7f0451ba68d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:08.069 7f0457fca6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 09/04/2025 05:13:08.075 7f0457fca6d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 09/04/2025 05:13:08.093 7f0457fca6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4013-3.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4013-3..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4013-3..tftp_queue', 'device_manager_ztp.ztp_queue'] 09/04/2025 05:13:11.683 7f0451151828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:13:11.685 7f0451151828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:13:11.685 7f0451151828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:13:11.685 7f0451151828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:11.685 7f0451151828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 09/04/2025 05:13:11.685 7f0451151828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:11.685 7f0451151828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:13:11.686 7f0451151828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:13:11.686 7f0451151828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:11.686 7f0451151828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 09/04/2025 05:13:11.686 7f0451151828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:11.686 7f0451151828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:11.686 7f0451151828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:13:11.686 7f0451151828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:13.532 7f0451151240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:13.533 7f0451151240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:13:13.534 7f0451151240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:15.685 7f0451200f60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:13:15.686 7f0451200f60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:13:15.687 7f0451200f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:13:15.687 7f0451200f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:15.687 7f0451200f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 05:13:15.687 7f0451200f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:15.687 7f0451200f60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:13:15.687 7f0451200f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:13:15.687 7f0451200f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:15.687 7f0451200f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:13:15.687 7f0451200f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:15.688 7f0451200f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:15.688 7f0451200f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:13:15.688 7f0451200f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:19.687 7f0451219240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:13:19.688 7f0451219240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:13:19.689 7f0451219240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:13:19.689 7f0451219240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:19.689 7f0451219240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 05:13:19.689 7f0451219240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:19.689 7f0451219240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:13:19.690 7f0451219240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:13:19.690 7f0451219240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:19.690 7f0451219240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:13:19.690 7f0451219240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:19.690 7f0451219240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:19.690 7f0451219240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:13:19.690 7f0451219240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:23.690 7f0456c84eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:13:23.691 7f0456c84eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:13:23.692 7f0456c84eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:13:23.692 7f0456c84eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:23.692 7f0456c84eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 05:13:23.692 7f0456c84eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:23.692 7f0456c84eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:13:23.693 7f0456c84eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:13:23.693 7f0456c84eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:23.693 7f0456c84eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:13:23.693 7f0456c84eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:23.693 7f0456c84eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:23.693 7f0456c84eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:13:23.693 7f0456c84eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:27.693 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:13:27.694 7f0451136eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:13:27.695 7f0451136eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:13:27.695 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:27.695 7f0451136eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 05:13:27.695 7f0451136eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:27.695 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:13:27.695 7f0451136eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:13:27.695 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:27.696 7f0451136eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:13:27.696 7f0451136eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:27.696 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:27.696 7f0451136eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:13:27.696 7f0451136eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:31.696 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:13:31.697 7f0451136eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:13:31.697 7f0451136eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:13:31.697 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:31.698 7f0451136eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 05:13:31.698 7f0451136eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:31.698 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:13:31.698 7f0451136eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:13:31.699 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:31.699 7f0451136eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:13:31.699 7f0451136eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:31.699 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:31.699 7f0451136eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:13:31.699 7f0451136eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:35.698 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:13:35.700 7f0451136eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:13:35.700 7f0451136eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:13:35.700 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:35.700 7f0451136eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 05:13:35.700 7f0451136eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:35.700 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:13:35.700 7f0451136eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:13:35.700 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:35.701 7f0451136eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:13:35.701 7f0451136eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:35.701 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:35.701 7f0451136eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:13:35.701 7f0451136eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:39.700 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:13:39.702 7f0451136eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:13:39.702 7f0451136eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:13:39.702 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:39.702 7f0451136eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 05:13:39.702 7f0451136eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:39.702 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:13:39.703 7f0451136eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:13:39.703 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:39.703 7f0451136eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:13:39.703 7f0451136eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:39.703 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:39.703 7f0451136eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:13:39.703 7f0451136eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:43.702 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:13:43.704 7f0451136eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:13:43.704 7f0451136eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:13:43.704 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:43.704 7f0451136eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 05:13:43.704 7f0451136eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:43.704 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:13:43.704 7f0451136eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:13:43.705 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:43.705 7f0451136eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:13:43.705 7f0451136eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:43.705 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:43.705 7f0451136eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:13:43.705 7f0451136eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:47.705 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:13:47.707 7f0451136eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:13:47.707 7f0451136eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:13:47.707 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:47.707 7f0451136eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 05:13:47.708 7f0451136eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:47.708 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:13:47.708 7f0451136eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:13:47.708 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:47.708 7f0451136eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:13:47.708 7f0451136eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:47.708 7f0451136eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:47.708 7f0451136eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:13:47.708 7f0451136eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:51.708 7f0452402630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:13:51.710 7f0452402630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:13:51.710 7f0452402630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:13:51.710 7f0452402630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:51.710 7f0452402630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 05:13:51.710 7f0452402630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:51.710 7f0452402630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:13:51.711 7f0452402630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:13:51.711 7f0452402630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:51.711 7f0452402630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:13:51.711 7f0452402630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:51.711 7f0452402630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:51.711 7f0452402630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:13:51.711 7f0452402630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:55.711 7f0452402630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:13:55.712 7f0452402630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:13:55.712 7f0452402630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:13:55.712 7f0452402630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:55.712 7f0452402630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 05:13:55.713 7f0452402630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:55.713 7f0452402630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:13:55.713 7f0452402630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:13:55.713 7f0452402630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:55.713 7f0452402630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:13:55.713 7f0452402630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:55.713 7f0452402630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:55.713 7f0452402630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:13:55.713 7f0452402630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:59.713 7f0452402630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:13:59.715 7f0452402630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:13:59.715 7f0452402630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:13:59.716 7f0452402630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:59.716 7f0452402630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 05:13:59.716 7f0452402630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:13:59.716 7f0452402630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:13:59.716 7f0452402630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:13:59.716 7f0452402630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:59.716 7f0452402630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:13:59.716 7f0452402630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:59.716 7f0452402630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:13:59.717 7f0452402630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:13:59.717 7f0452402630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:03.716 7f0456c7e240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:14:03.718 7f0456c7e240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:14:03.718 7f0456c7e240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:14:03.718 7f0456c7e240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:03.718 7f0456c7e240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 05:14:03.718 7f0456c7e240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:03.718 7f0456c7e240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:14:03.718 7f0456c7e240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:14:03.719 7f0456c7e240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:03.719 7f0456c7e240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:14:03.719 7f0456c7e240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:03.719 7f0456c7e240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:03.719 7f0456c7e240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:14:03.719 7f0456c7e240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:07.718 7f04553d7240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:14:07.719 7f04553d7240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:14:07.720 7f04553d7240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:14:07.720 7f04553d7240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:07.720 7f04553d7240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 05:14:07.720 7f04553d7240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:07.720 7f04553d7240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:14:07.720 7f04553d7240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:14:07.720 7f04553d7240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:07.720 7f04553d7240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:14:07.720 7f04553d7240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:07.721 7f04553d7240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:07.721 7f04553d7240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:14:07.721 7f04553d7240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:11.720 7f04592f8588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:14:11.722 7f04592f8588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:14:11.723 7f04592f8588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:14:11.723 7f04592f8588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:11.723 7f04592f8588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 05:14:11.723 7f04592f8588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:11.723 7f04592f8588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:14:11.723 7f04592f8588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:14:11.723 7f04592f8588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:11.724 7f04592f8588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:14:11.724 7f04592f8588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:11.724 7f04592f8588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:11.724 7f04592f8588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:14:11.724 7f04592f8588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:15.723 7f04525640f0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:14:15.724 7f04525640f0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:14:15.725 7f04525640f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:14:15.725 7f04525640f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:15.725 7f04525640f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 05:14:15.725 7f04525640f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:15.725 7f04525640f0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:14:15.725 7f04525640f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:14:15.726 7f04525640f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:15.726 7f04525640f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:14:15.726 7f04525640f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:15.726 7f04525640f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:15.726 7f04525640f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:14:15.726 7f04525640f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:19.725 7f0452564048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:14:19.726 7f0452564048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:14:19.727 7f0452564048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:14:19.727 7f0452564048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:19.727 7f0452564048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 05:14:19.727 7f0452564048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:19.727 7f0452564048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:14:19.727 7f0452564048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:14:19.727 7f0452564048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:19.728 7f0452564048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:14:19.728 7f0452564048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:19.728 7f0452564048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:19.728 7f0452564048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:14:19.728 7f0452564048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:23.728 7f0452564048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:14:23.730 7f0452564048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:14:23.730 7f0452564048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:14:23.730 7f0452564048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:23.730 7f0452564048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 05:14:23.730 7f0452564048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:23.730 7f0452564048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:14:23.731 7f0452564048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:14:23.731 7f0452564048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:23.731 7f0452564048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:14:23.731 7f0452564048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:23.731 7f0452564048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:23.731 7f0452564048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:14:23.731 7f0452564048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:27.730 7f04510c3e10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:14:27.732 7f04510c3e10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:14:27.732 7f04510c3e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:14:27.732 7f04510c3e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:27.732 7f04510c3e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 05:14:27.732 7f04510c3e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:27.732 7f04510c3e10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:14:27.732 7f04510c3e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:14:27.733 7f04510c3e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:27.733 7f04510c3e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:14:27.733 7f04510c3e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:27.733 7f04510c3e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:27.733 7f04510c3e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:14:27.733 7f04510c3e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:31.732 7f0451188630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:14:31.733 7f0451188630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:14:31.733 7f0451188630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:14:31.734 7f0451188630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:31.734 7f0451188630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 05:14:31.734 7f0451188630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:31.734 7f0451188630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:14:31.734 7f0451188630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:14:31.734 7f0451188630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:31.734 7f0451188630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:14:31.734 7f0451188630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:31.734 7f0451188630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:31.735 7f0451188630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:14:31.735 7f0451188630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:34.240 7f045b1038c8 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 09/04/2025 05:14:34.240 7f045b1038c8 [contrail-device-manager] [WARNING]: initializing cassandra cluster 09/04/2025 05:14:34.241 7f045b1038c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/04/2025 05:14:34.241 7f0451188198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:34.241 7f0451188198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/04/2025 05:14:34.241 7f0451188198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:34.242 7f045b1038c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/04/2025 05:14:34.272 7f045b1038c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/04/2025 05:14:34.274 7f0451301978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:34.274 7f0451301978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Initializing description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/04/2025 05:14:34.274 7f0451301978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:34.281 7f045b1038c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 09/04/2025 05:14:34.281 7f0451301780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:34.281 7f0451301780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/04/2025 05:14:34.281 7f0451301780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:34.293 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/04/2025 05:14:34.974 7f045b1038c8 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace', created 09/04/2025 05:14:34.986 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/04/2025 05:14:35.004 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/04/2025 05:14:35.734 7f0451106cc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:14:35.736 7f0451106cc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:14:35.736 7f0451106cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:14:35.736 7f0451106cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:35.736 7f0451106cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/04/2025 05:14:35.736 7f0451106cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:35.736 7f0451106cc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:14:35.737 7f0451106cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:14:35.737 7f0451106cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:35.737 7f0451106cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/04/2025 05:14:35.737 7f0451106cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:35.737 7f0451106cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:35.737 7f0451106cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:14:35.737 7f0451106cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:35.904 7f045b1038c8 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None} 09/04/2025 05:14:35.909 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/04/2025 05:14:35.909 7f045b1038c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 09/04/2025 05:14:36.021 7f045b1038c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 09/04/2025 05:14:36.029 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/04/2025 05:14:36.843 7f045b1038c8 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table', created 09/04/2025 05:14:36.856 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/04/2025 05:14:36.873 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/04/2025 05:14:37.971 7f045b1038c8 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table' fixed with properties {'gc_grace_seconds': 864000} 09/04/2025 05:14:37.975 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/04/2025 05:14:37.983 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/04/2025 05:14:38.786 7f045b1038c8 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table', created 09/04/2025 05:14:38.790 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/04/2025 05:14:38.794 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/04/2025 05:14:39.736 7f0451bae438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:14:39.738 7f0451bae438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:14:39.738 7f0451bae438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:14:39.739 7f0451bae438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:39.739 7f0451bae438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/04/2025 05:14:39.739 7f0451bae438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:39.739 7f0451bae438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:14:39.739 7f0451bae438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:14:39.739 7f0451bae438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:39.739 7f0451bae438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/04/2025 05:14:39.739 7f0451bae438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:39.740 7f0451bae438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:39.740 7f0451bae438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:14:39.740 7f0451bae438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:39.910 7f045b1038c8 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table' fixed with properties {'gc_grace_seconds': 864000} 09/04/2025 05:14:39.916 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/04/2025 05:14:39.922 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/04/2025 05:14:40.934 7f045b1038c8 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table', created 09/04/2025 05:14:40.949 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/04/2025 05:14:40.967 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/04/2025 05:14:41.900 7f045b1038c8 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table' fixed with properties {'gc_grace_seconds': 864000} 09/04/2025 05:14:41.904 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/04/2025 05:14:41.910 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/04/2025 05:14:43.739 7f04510aff60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:14:43.741 7f04510aff60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:14:43.741 7f04510aff60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:14:43.741 7f04510aff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:43.741 7f04510aff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/04/2025 05:14:43.742 7f04510aff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:43.742 7f04510aff60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:14:43.742 7f04510aff60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:14:43.742 7f04510aff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:43.742 7f04510aff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/04/2025 05:14:43.743 7f04510aff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:43.743 7f04510aff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:43.743 7f04510aff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:14:43.743 7f04510aff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:43.797 7f045b1038c8 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table', created 09/04/2025 05:14:43.802 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/04/2025 05:14:43.810 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/04/2025 05:14:44.904 7f045b1038c8 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table' fixed with properties {'gc_grace_seconds': 864000} 09/04/2025 05:14:44.908 7f045b1038c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/04/2025 05:14:44.908 7f04510d9e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:44.908 7f04510d9e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:14:44.908 7f04510d9e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:44.960 7f045b1038c8 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.254:5673//', 'pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/04/2025 05:14:44.961 7f045b1038c8 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 09/04/2025 05:14:44.963 7f0451098198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:44.963 7f0451098198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Initializing description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 09/04/2025 05:14:44.963 7f0451098198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:44.964 7f0451098198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:44.964 7f0451098198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 09/04/2025 05:14:44.964 7f0451098198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:44.970 7f045b1038c8 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/04/2025 05:14:44.971 7f045112d6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:44.971 7f045112d6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:14:44.971 7f045112d6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:45.026 7f045b1038c8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 09/04/2025 05:14:45.027 7f0451bb7b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:45.027 7f0451bb7b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 09/04/2025 05:14:45.027 7f0451bb7b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:45.047 7f045b1038c8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/04/2025 05:14:45.048 7f04510aac18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:45.049 7f04510aac18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:14:45.049 7f04510aac18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:45.051 7f04511f2978 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 09/04/2025 05:14:45.052 7f04510afb70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:45.053 7f04510afb70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 09/04/2025 05:14:45.053 7f04510afb70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:45.058 7f04511f2978 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/04/2025 05:14:45.059 7f0451122198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:45.059 7f0451122198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:14:45.059 7f0451122198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:47.742 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:14:47.743 7f045110a048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:14:47.744 7f045110a048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:14:47.744 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:47.744 7f045110a048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 05:14:47.744 7f045110a048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:47.745 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:14:47.746 7f045110a048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:14:47.746 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:47.746 7f045110a048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:14:47.746 7f045110a048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:47.746 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:47.747 7f045110a048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:14:47.747 7f045110a048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:51.745 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:14:51.747 7f045110a048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:14:51.747 7f045110a048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:14:51.747 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:51.747 7f045110a048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 05:14:51.747 7f045110a048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:51.747 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:14:51.748 7f045110a048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:14:51.748 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:51.748 7f045110a048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:14:51.748 7f045110a048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:51.748 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:51.748 7f045110a048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:14:51.748 7f045110a048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:55.748 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:14:55.749 7f045110a048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:14:55.749 7f045110a048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:14:55.749 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:55.749 7f045110a048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 05:14:55.749 7f045110a048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:55.750 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:14:55.750 7f045110a048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:14:55.750 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:55.750 7f045110a048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:14:55.750 7f045110a048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:55.750 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:55.750 7f045110a048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:14:55.750 7f045110a048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:59.750 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:14:59.752 7f045110a048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:14:59.752 7f045110a048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:14:59.752 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:59.753 7f045110a048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 05:14:59.753 7f045110a048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:14:59.753 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:14:59.753 7f045110a048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:14:59.753 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:59.753 7f045110a048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:14:59.753 7f045110a048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:59.754 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:14:59.754 7f045110a048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:14:59.754 7f045110a048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:15:03.754 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:15:03.755 7f045110a048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:15:03.755 7f045110a048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:15:03.755 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:15:03.755 7f045110a048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 05:15:03.755 7f045110a048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:15:03.755 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:15:03.756 7f045110a048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:15:03.756 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:15:03.756 7f045110a048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:15:03.756 7f045110a048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:15:03.756 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:15:03.756 7f045110a048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:15:03.756 7f045110a048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:15:07.756 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:15:07.758 7f045110a048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 05:15:07.758 7f045110a048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:15:07.758 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:15:07.758 7f045110a048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 05:15:07.758 7f045110a048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:15:07.758 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 05:15:07.759 7f045110a048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 05:15:07.759 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:15:07.759 7f045110a048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 05:15:07.759 7f045110a048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:15:07.759 7f045110a048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:15:07.760 7f045110a048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 http_port = 8096 start_time = 1756962767673643 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 05:15:07.760 7f045110a048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 05:15:09.667 7f04511f2630 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4013-1.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.254', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4013-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4013-1.'], 'id_perms': {'created': '2025-09-04T05:15:09.659426', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-04T05:15:09.659426', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13184158297827324010, 'uuid_mslong': 11838587474533370812}}, 'parent_type': 'global-system-config', 'parent_uuid': '65c094ed-5aca-4143-aa5b-4c2afbfec188', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'a44b1c95-a5f3-4bbc-b6f7-8a1e8484306a'}, 'oper': 'CREATE', 'request-id': 'req-48b24ae5-5783-4b9a-9cbf-81d9052ce8fa', 'type': 'analytics_node', 'uuid': 'a44b1c95-a5f3-4bbc-b6f7-8a1e8484306a'} 09/04/2025 05:15:10.132 7f04511f2630 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4013-3.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.50', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4013-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4013-3.'], 'id_perms': {'created': '2025-09-04T05:15:10.104928', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-04T05:15:10.104928', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13417016445435215185, 'uuid_mslong': 6112756768802032196}}, 'parent_type': 'global-system-config', 'parent_uuid': '65c094ed-5aca-4143-aa5b-4c2afbfec188', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '54d4dfe4-facf-4a44-ba32-d15ee7bd7d51'}, 'oper': 'CREATE', 'request-id': 'req-31fb29db-fcca-40cd-a490-fceb51c5e574', 'type': 'analytics_node', 'uuid': '54d4dfe4-facf-4a44-ba32-d15ee7bd7d51'} 09/04/2025 05:15:10.237 7f04511f2630 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4013-2.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.38', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4013-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4013-2.'], 'id_perms': {'created': '2025-09-04T05:15:10.215663', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-04T05:15:10.215663', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12272495807619536806, 'uuid_mslong': 15695948060925382557}}, 'parent_type': 'global-system-config', 'parent_uuid': '65c094ed-5aca-4143-aa5b-4c2afbfec188', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'd9d33599-c870-4f9d-aa50-a9ea20a217a6'}, 'oper': 'CREATE', 'request-id': 'req-d31ea81d-b43f-4703-aac2-7e1520c40cfb', 'type': 'analytics_node', 'uuid': 'd9d33599-c870-4f9d-aa50-a9ea20a217a6'} 09/04/2025 05:15:11.758 7f0451025390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 05:15:11.762 7f0451025390 [contrail-device-manager] [INFO]: Session Event: TCP Connected 09/04/2025 05:15:11.762 7f0451025390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 05:15:11.762 7f0451025390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:15:11.762 7f0451025390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4013-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 05:15:11.762 7f0451025390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 05:15:11.762 7f0451025390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/04/2025 05:15:11.763 7f0451025390 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19 09/04/2025 05:15:11.763 7f0451025390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/04/2025 05:15:11.764 7f0451025390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/04/2025 05:15:11.764 7f0451025390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/04/2025 05:15:11.764 7f0451026ac8 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/04/2025 05:15:11.766 7f0451026ac8 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/04/2025 05:15:21.012 7f0451026978 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001016 09/04/2025 05:15:21.012 7f0451026978 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 09/04/2025 05:15:21.012 7f04510b4b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 09/04/2025 05:15:21.013 7f04510b4b70 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 0 09/04/2025 05:15:21.013 7f04510b4b70 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/04/2025 05:15:21.013 7f04510b4b70 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/04/2025 05:15:21.013 7f04510b4b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 09/04/2025 05:15:21.013 7f04510b4ac8 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/04/2025 05:15:21.014 7f04510b4ac8 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/04/2025 05:15:28.398 7f04511f2630 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4013-3.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.50', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4013-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4013-3.'], 'id_perms': {'created': '2025-09-04T05:15:28.382000', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-04T05:15:28.382000', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11948363787058382520, 'uuid_mslong': 2872630284261474685}}, 'parent_type': 'global-system-config', 'parent_uuid': '65c094ed-5aca-4143-aa5b-4c2afbfec188', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '27dda206-106f-417d-a5d1-1d8f26c186b8'}, 'oper': 'CREATE', 'request-id': 'req-397bb14c-b285-41e0-a672-2a888dfeed7c', 'type': 'analytics_alarm_node', 'uuid': '27dda206-106f-417d-a5d1-1d8f26c186b8'} 09/04/2025 05:15:28.520 7f04511f2630 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4013-2.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.38', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4013-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4013-2.'], 'id_perms': {'created': '2025-09-04T05:15:28.504238', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-04T05:15:28.504238', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9235533828636732107, 'uuid_mslong': 2884956224829932718}}, 'parent_type': 'global-system-config', 'parent_uuid': '65c094ed-5aca-4143-aa5b-4c2afbfec188', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '28096c66-bd8a-44ae-802b-3515cefeeacb'}, 'oper': 'CREATE', 'request-id': 'req-255c0a75-64a8-4a7c-b3ec-96e844675d75', 'type': 'analytics_alarm_node', 'uuid': '28096c66-bd8a-44ae-802b-3515cefeeacb'} 09/04/2025 05:15:28.610 7f04511f2630 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4013-1.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.254', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4013-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4013-1.'], 'id_perms': {'created': '2025-09-04T05:15:28.597021', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-04T05:15:28.597021', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11848057690453468190, 'uuid_mslong': 5985099291300872813}}, 'parent_type': 'global-system-config', 'parent_uuid': '65c094ed-5aca-4143-aa5b-4c2afbfec188', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '530f5818-536e-426d-a46c-c1b21dd0641e'}, 'oper': 'CREATE', 'request-id': 'req-d6a1e7d0-d69f-4c81-9698-28dccebb1908', 'type': 'analytics_alarm_node', 'uuid': '530f5818-536e-426d-a46c-c1b21dd0641e'} 09/04/2025 05:15:32.108 7f0451026978 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001296 09/04/2025 05:15:32.109 7f0451026978 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 09/04/2025 05:15:32.110 7f0458146eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 09/04/2025 05:15:32.110 7f0458146eb8 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 2 09/04/2025 05:15:32.110 7f0458146eb8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/04/2025 05:15:32.110 7f0458146eb8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/04/2025 05:15:38.124 7f04511f2630 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4013-1.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.254', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4013-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4013-1.'], 'id_perms': {'created': '2025-09-04T05:15:38.108949', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-04T05:15:38.108949', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10533426732142312891, 'uuid_mslong': 16648920875087843164}}, 'parent_type': 'global-system-config', 'parent_uuid': '65c094ed-5aca-4143-aa5b-4c2afbfec188', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'e70cd953-7d32-4f5c-922e-3fcdecf23dbb'}, 'oper': 'CREATE', 'request-id': 'req-61a2a71f-3644-4049-b65c-05937cb17f28', 'type': 'analytics_snmp_node', 'uuid': 'e70cd953-7d32-4f5c-922e-3fcdecf23dbb'} 09/04/2025 05:15:38.151 7f04511f2630 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4013-2.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.38', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4013-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4013-2.'], 'id_perms': {'created': '2025-09-04T05:15:38.139456', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-04T05:15:38.139456', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11077678382024518825, 'uuid_mslong': 5567981425778313394}}, 'parent_type': 'global-system-config', 'parent_uuid': '65c094ed-5aca-4143-aa5b-4c2afbfec188', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '4d45719b-377d-48b2-99bb-d1cbe0986ca9'}, 'oper': 'CREATE', 'request-id': 'req-97504ab1-75ab-4df6-a96e-3c5c4da1995f', 'type': 'analytics_snmp_node', 'uuid': '4d45719b-377d-48b2-99bb-d1cbe0986ca9'} 09/04/2025 05:15:38.171 7f04511f2630 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4013-3.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.50', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4013-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4013-3.'], 'id_perms': {'created': '2025-09-04T05:15:38.144071', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-04T05:15:38.144071', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13260910381944517744, 'uuid_mslong': 18053386661978656644}}, 'parent_type': 'global-system-config', 'parent_uuid': '65c094ed-5aca-4143-aa5b-4c2afbfec188', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'fa8a8384-a403-4f84-b808-37bbaa17b470'}, 'oper': 'CREATE', 'request-id': 'req-bf55c1cf-b2e4-4d25-bc22-dd079e49a68e', 'type': 'analytics_snmp_node', 'uuid': 'fa8a8384-a403-4f84-b808-37bbaa17b470'} 09/04/2025 05:17:20.368 7f04511f2630 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-47cff42d-deac-4d19-993b-1622a18f24f0', 'type': 'global_system_config', 'uuid': '65c094ed-5aca-4143-aa5b-4c2afbfec188'} 09/04/2025 05:17:20.496 7f04511f2630 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4013-2'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-4013-2', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4013-2'], 'id_perms': {'created': '2025-09-04T05:17:20.487094', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-04T05:17:20.487094', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13278167948437481015, 'uuid_mslong': 8995008935372080019}}, 'parent_type': 'global-system-config', 'parent_uuid': '65c094ed-5aca-4143-aa5b-4c2afbfec188', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '7cd4b0f8-1851-4393-b845-8765fa97e637', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.193'}, 'oper': 'CREATE', 'request-id': 'req-706a0898-f569-4b15-a4b6-01d5671dcbf7', 'type': 'virtual_router', 'uuid': '7cd4b0f8-1851-4393-b845-8765fa97e637'} 09/04/2025 05:17:20.839 7f04511f2630 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4013-2', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4013-2', 'vhost0'], 'id_perms': {'created': '2025-09-04T05:17:20.814875', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-04T05:17:20.814875', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10807943526900334900, 'uuid_mslong': 6674531541079182929}}, 'parent_type': 'virtual-router', 'parent_uuid': '7cd4b0f8-1851-4393-b845-8765fa97e637', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '5ca0b312-cc2a-4e51-95fd-875ff1608134', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:5c:a0:b3:12:cc']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': '2727b4cf-6120-4db2-99cb-50a5809fbdcf'}]}, 'oper': 'CREATE', 'request-id': 'req-a5fa6a46-9739-447a-9c2b-9d214dfebdd5', 'type': 'virtual_machine_interface', 'uuid': '5ca0b312-cc2a-4e51-95fd-875ff1608134'} 09/04/2025 05:17:20.877 7f04511f2630 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4013-2', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-a5fa6a46-9739-447a-9c2b-9d214dfebdd5', 'type': 'virtual_machine_interface', 'uuid': '5ca0b312-cc2a-4e51-95fd-875ff1608134'} 09/04/2025 05:17:21.385 7f04511f2630 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-e1d38688-0c49-4944-9698-1b5800efbb75', 'type': 'access_control_list', 'uuid': 'ab8718ee-0e60-4209-bcb8-57d74373f51c'} 09/04/2025 05:17:21.676 7f04511f2630 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4013-1'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-4013-1', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4013-1'], 'id_perms': {'created': '2025-09-04T05:17:21.666070', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-04T05:17:21.666070', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10947160260048703893, 'uuid_mslong': 4679461470752229219}}, 'parent_type': 'global-system-config', 'parent_uuid': '65c094ed-5aca-4143-aa5b-4c2afbfec188', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '40f0c96a-3050-4763-97ec-20418779d195', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.25'}, 'oper': 'CREATE', 'request-id': 'req-59183591-b39f-49dc-b95c-42b79bb6d253', 'type': 'virtual_router', 'uuid': '40f0c96a-3050-4763-97ec-20418779d195'} 09/04/2025 05:17:22.040 7f04511f2630 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4013-1', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4013-1', 'vhost0'], 'id_perms': {'created': '2025-09-04T05:17:22.016479', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-04T05:17:22.016479', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13414568029919353441, 'uuid_mslong': 3092787255852354653}}, 'parent_type': 'virtual-router', 'parent_uuid': '40f0c96a-3050-4763-97ec-20418779d195', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '2aebc99f-76ef-4c5d-ba2a-1e8cbea79a61', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:2a:eb:c9:9f:76']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': '2727b4cf-6120-4db2-99cb-50a5809fbdcf'}]}, 'oper': 'CREATE', 'request-id': 'req-bb2630cb-14d5-4629-ae04-791335940048', 'type': 'virtual_machine_interface', 'uuid': '2aebc99f-76ef-4c5d-ba2a-1e8cbea79a61'} 09/04/2025 05:17:22.073 7f04511f2630 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4013-1', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-bb2630cb-14d5-4629-ae04-791335940048', 'type': 'virtual_machine_interface', 'uuid': '2aebc99f-76ef-4c5d-ba2a-1e8cbea79a61'} 09/04/2025 05:37:08.183 7f04510b4780 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map