11/03/2025 05:31:37.467 7f346ec83348 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 11/03/2025 05:31:37.467 7f346ec83348 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 11/03/2025 05:31:37.467 7f346ec83348 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 11/03/2025 05:31:37.467 7f346636d048 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 11/03/2025 05:31:37.468 7f346636d048 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 11/03/2025 05:31:37.468 7f346636d048 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/03/2025 05:31:37.468 7f346636d048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 11/03/2025 05:31:37.468 7f346636d048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:37.468 7f346636d048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/03/2025 05:31:37.468 7f346636d048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:37.469 7f346636d048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:37.469 7f346636d048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 start_time = 1762147897468564 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 11/03/2025 05:31:37.469 7f346636d048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:37.512 7f346ec83348 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.48:5673//', 'pyamqp://guest:guest@10.0.0.49:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 11/03/2025 05:31:37.520 7f346ec83348 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 11/03/2025 05:31:37.521 7f34662e32e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:37.521 7f34662e32e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/03/2025 05:31:37.521 7f34662e32e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:37.521 7f34662e32e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:37.521 7f34662e32e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/03/2025 05:31:37.521 7f34662e32e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:37.530 7f346ec83348 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 11/03/2025 05:31:37.530 7f34662e3828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:37.530 7f34662e3828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/03/2025 05:31:37.530 7f34662e3828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:37.545 7f346ec83348 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 11/03/2025 05:31:37.545 7f34662cc4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:37.545 7f34662cc4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/03/2025 05:31:37.546 7f34662cc4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:37.552 7f346ec83348 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 11/03/2025 05:31:37.552 7f3474b6e978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:37.552 7f3474b6e978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/03/2025 05:31:37.552 7f3474b6e978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:37.848 7f346ec83348 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 11/03/2025 05:31:37.848 7f346ec83348 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 11/03/2025 05:31:37.848 7f346ec83348 [contrail-device-manager] [WARNING]: initializing cassandra cluster 11/03/2025 05:31:37.848 7f346ec83348 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 11/03/2025 05:31:37.849 7f346bbf46d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 11/03/2025 05:31:37.849 7f346bbf46d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/03/2025 05:31:37.849 7f34662e3ac8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 11/03/2025 05:31:37.849 7f34662e3ac8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 11/03/2025 05:31:37.850 7f346615c048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:37.850 7f346615c048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/03/2025 05:31:37.850 7f346615c048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:37.851 7f346ec83348 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 11/03/2025 05:31:37.855 7f346bbf46d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/03/2025 05:31:37.855 7f346bbf46d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 11/03/2025 05:31:37.855 7f34662e3ac8 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 11/03/2025 05:31:37.884 7f346ec83348 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 11/03/2025 05:31:37.940 7f34660ec438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:37.943 7f34660ec438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 11/03/2025 05:31:37.943 7f34660ec438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:37.951 7f346ec83348 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 11/03/2025 05:31:37.951 7f3466074438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:37.951 7f3466074438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/03/2025 05:31:37.951 7f3466074438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:38.963 7f346ec83348 [contrail-device-manager] [WARNING]: Waiting for keyspace config_db_uuid to be created 11/03/2025 05:31:39.989 7f346ec83348 [contrail-device-manager] [WARNING]: Waiting for keyspace config_db_uuid to be created 11/03/2025 05:31:39.995 7f346ec83348 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 11/03/2025 05:31:40.051 7f346ec83348 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 11/03/2025 05:31:41.289 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/03/2025 05:31:41.469 7f346603bb70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:31:41.470 7f346603bb70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:31:41.470 7f346603bb70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:31:41.470 7f346603bb70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:31:41.470 7f346603bb70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 11/03/2025 05:31:41.470 7f346603bb70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:31:41.470 7f346603bb70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:31:41.471 7f346603bb70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:31:41.471 7f346603bb70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:41.471 7f346603bb70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 11/03/2025 05:31:41.471 7f346603bb70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:41.471 7f346603bb70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:41.471 7f346603bb70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:31:41.471 7f346603bb70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:42.239 7f346ec83348 [contrail-device-manager] [WARNING]: table 'obj_uuid_table', created 11/03/2025 05:31:42.244 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/03/2025 05:31:43.243 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/03/2025 05:31:44.216 7f346ec83348 [contrail-device-manager] [WARNING]: table 'obj_uuid_table' fixed with properties {'gc_grace_seconds': 864000} 11/03/2025 05:31:44.220 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/03/2025 05:31:45.470 7f346602c828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:31:45.471 7f346602c828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:31:45.471 7f346602c828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:31:45.471 7f346602c828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:31:45.472 7f346602c828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 11/03/2025 05:31:45.472 7f346602c828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:31:45.472 7f346602c828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:31:45.472 7f346602c828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:31:45.472 7f346602c828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:45.472 7f346602c828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 11/03/2025 05:31:45.472 7f346602c828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:45.472 7f346602c828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:45.472 7f346602c828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:31:45.472 7f346602c828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:46.276 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/03/2025 05:31:47.216 7f346ec83348 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table', created 11/03/2025 05:31:47.221 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/03/2025 05:31:49.232 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/03/2025 05:31:49.473 7f346602ca20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:31:49.474 7f346602ca20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:31:49.474 7f346602ca20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:31:49.474 7f346602ca20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:31:49.475 7f346602ca20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 11/03/2025 05:31:49.475 7f346602ca20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:31:49.475 7f346602ca20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:31:49.475 7f346602ca20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:31:49.476 7f346602ca20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:49.476 7f346602ca20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 11/03/2025 05:31:49.476 7f346602ca20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:49.476 7f346602ca20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:49.476 7f346602ca20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:31:49.476 7f346602ca20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:49.733 7f346ec83348 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table' fixed with properties {'gc_grace_seconds': 864000} 11/03/2025 05:31:49.737 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/03/2025 05:31:51.741 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/03/2025 05:31:51.744 7f346ec83348 [contrail-device-manager] [WARNING]: table 'obj_shared_table', created 11/03/2025 05:31:51.747 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/03/2025 05:31:53.476 7f346603b2e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:31:53.477 7f346603b2e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:31:53.477 7f346603b2e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:31:53.477 7f346603b2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:31:53.477 7f346603b2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 11/03/2025 05:31:53.477 7f346603b2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:31:53.477 7f346603b2e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:31:53.478 7f346603b2e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:31:53.478 7f346603b2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:53.478 7f346603b2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 11/03/2025 05:31:53.478 7f346603b2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:53.478 7f346603b2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:53.478 7f346603b2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:31:53.478 7f346603b2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:54.173 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/03/2025 05:31:55.323 7f346ec83348 [contrail-device-manager] [WARNING]: table 'obj_shared_table' fixed with properties {'gc_grace_seconds': 864000} 11/03/2025 05:31:55.330 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/03/2025 05:31:55.331 7f346602ca20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:55.332 7f346602ca20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:31:55.332 7f346602ca20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:55.346 7f346ec83348 [contrail-device-manager] [ERROR]: Error in db walk read 'type' 11/03/2025 05:31:55.347 7f34657dbac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:55.347 7f34657dbac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Zookeeper:Zookeeper[] connection down >>, ] >> 11/03/2025 05:31:55.347 7f34657dbac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:55.361 7f34657dbac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:55.361 7f34657dbac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:31:55.361 7f34657dbac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:55.361 7f346ec83348 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 11/03/2025 05:31:55.361 7f346ec83348 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 11/03/2025 05:31:55.362 7f346bbf46d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/03/2025 05:31:55.363 7f346ec83348 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4601-2. 11/03/2025 05:31:55.368 7f346bbf46d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/03/2025 05:31:55.445 7f346bbf46d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4601-2.'] 11/03/2025 05:31:55.458 7f346bbf46d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/03/2025 05:31:55.463 7f346bbf46d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/03/2025 05:31:55.508 7f346bbf46d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4601-2.'] 11/03/2025 05:31:55.873 7f346ec83348 [contrail-device-manager] [DEBUG]: Fetching all containers 11/03/2025 05:31:55.910 7f346ec83348 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 11/03/2025 05:31:56.062 7f346ec83348 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4601-2..config_queue 11/03/2025 05:31:56.063 7f346ec83348 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4601-2..tftp_queue 11/03/2025 05:31:56.523 7f346bbf46d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/03/2025 05:31:56.529 7f346bbf46d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/03/2025 05:31:56.564 7f346ec83348 [contrail-device-manager] [WARNING]: Process 138 prepared to run in Full mode... 11/03/2025 05:31:56.564 7f346ec83348 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 11/03/2025 05:31:56.564 7f346ec83348 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 11/03/2025 05:31:56.567 7f346ec83348 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page 11/03/2025 05:31:56.567 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 11/03/2025 05:31:56.568 7f346ec83348 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page 11/03/2025 05:31:56.568 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 11/03/2025 05:31:56.569 7f346ec83348 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 11/03/2025 05:31:56.569 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 11/03/2025 05:31:56.572 7f346ec83348 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page 11/03/2025 05:31:56.572 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 11/03/2025 05:31:56.572 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 11/03/2025 05:31:56.572 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 11/03/2025 05:31:56.572 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 11/03/2025 05:31:56.572 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 11/03/2025 05:31:56.572 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 11/03/2025 05:31:56.573 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 11/03/2025 05:31:56.573 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 11/03/2025 05:31:56.573 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 11/03/2025 05:31:56.573 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 11/03/2025 05:31:56.573 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 11/03/2025 05:31:56.573 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 11/03/2025 05:31:56.574 7f346ec83348 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 11/03/2025 05:31:56.574 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 11/03/2025 05:31:56.574 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 11/03/2025 05:31:56.574 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 11/03/2025 05:31:56.575 7f346ec83348 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 11/03/2025 05:31:56.575 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 11/03/2025 05:31:56.575 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 11/03/2025 05:31:56.575 7f346ec83348 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page 11/03/2025 05:31:56.575 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 11/03/2025 05:31:56.575 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 11/03/2025 05:31:56.575 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 11/03/2025 05:31:56.575 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 11/03/2025 05:31:56.576 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 11/03/2025 05:31:56.576 7f346ec83348 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page 11/03/2025 05:31:56.576 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 11/03/2025 05:31:56.576 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 11/03/2025 05:31:56.576 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 11/03/2025 05:31:56.576 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 11/03/2025 05:31:56.577 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 11/03/2025 05:31:56.577 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 11/03/2025 05:31:56.577 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 11/03/2025 05:31:56.577 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 11/03/2025 05:31:56.577 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 11/03/2025 05:31:56.577 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 11/03/2025 05:31:56.577 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 11/03/2025 05:31:56.577 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 11/03/2025 05:31:56.577 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 11/03/2025 05:31:56.577 7f346ec83348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 11/03/2025 05:31:56.580 7f346ec83348 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 11/03/2025 05:31:56.592 7f3465711ac8 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 11/03/2025 05:31:56.592 7f3465711ac8 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 11/03/2025 05:31:56.638 7f3464f12390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:56.638 7f3464f12390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 11/03/2025 05:31:56.638 7f3464f12390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:56.649 7f3464e52cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:56.649 7f3464e52cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 11/03/2025 05:31:56.649 7f3464e52cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:56.718 7f346bbf46d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4601-2.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4601-2..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4601-2..tftp_queue', 'device_manager_ztp.ztp_queue'] 11/03/2025 05:31:56.736 7f346bbf46d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/03/2025 05:31:56.741 7f346bbf46d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/03/2025 05:31:56.781 7f346bbf46d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4601-2.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4601-2..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4601-2..tftp_queue', 'device_manager_ztp.ztp_queue'] 11/03/2025 05:31:57.479 7f3464e4bcc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:31:57.480 7f3464e4bcc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:31:57.480 7f3464e4bcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:31:57.480 7f3464e4bcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:31:57.480 7f3464e4bcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 11/03/2025 05:31:57.480 7f3464e4bcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:31:57.480 7f3464e4bcc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:31:57.480 7f3464e4bcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:31:57.481 7f3464e4bcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:57.481 7f3464e4bcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 11/03/2025 05:31:57.481 7f3464e4bcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:57.481 7f3464e4bcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:31:57.481 7f3464e4bcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:31:57.481 7f3464e4bcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:01.480 7f3464d8d978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:32:01.482 7f3464d8d978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:32:01.482 7f3464d8d978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:32:01.482 7f3464d8d978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:01.482 7f3464d8d978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 11/03/2025 05:32:01.482 7f3464d8d978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:01.482 7f3464d8d978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:32:01.483 7f3464d8d978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:32:01.483 7f3464d8d978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:01.483 7f3464d8d978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 11/03/2025 05:32:01.483 7f3464d8d978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:01.483 7f3464d8d978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:01.483 7f3464d8d978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:32:01.483 7f3464d8d978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:02.785 7f3464d77978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:02.786 7f3464d77978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:32:02.786 7f3464d77978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:05.482 7f34662cc390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:32:05.484 7f34662cc390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:32:05.484 7f34662cc390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:32:05.484 7f34662cc390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:05.484 7f34662cc390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:32:05.484 7f34662cc390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:05.485 7f34662cc390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:32:05.485 7f34662cc390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:32:05.485 7f34662cc390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:05.485 7f34662cc390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:32:05.485 7f34662cc390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:05.485 7f34662cc390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:05.485 7f34662cc390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:32:05.485 7f34662cc390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:09.485 7f3464ca0ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:32:09.486 7f3464ca0ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:32:09.486 7f3464ca0ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:32:09.486 7f3464ca0ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:09.486 7f3464ca0ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:32:09.486 7f3464ca0ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:09.486 7f3464ca0ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:32:09.487 7f3464ca0ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:32:09.487 7f3464ca0ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:09.487 7f3464ca0ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:32:09.487 7f3464ca0ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:09.487 7f3464ca0ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:09.487 7f3464ca0ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:32:09.487 7f3464ca0ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:13.488 7f3464ca0ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:32:13.489 7f3464ca0ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:32:13.489 7f3464ca0ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:32:13.489 7f3464ca0ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:13.489 7f3464ca0ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:32:13.489 7f3464ca0ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:13.489 7f3464ca0ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:32:13.490 7f3464ca0ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:32:13.490 7f3464ca0ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:13.490 7f3464ca0ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:32:13.490 7f3464ca0ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:13.490 7f3464ca0ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:13.490 7f3464ca0ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:32:13.490 7f3464ca0ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:17.490 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:32:17.492 7f3464c9b048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:32:17.492 7f3464c9b048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:32:17.492 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:17.492 7f3464c9b048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:32:17.492 7f3464c9b048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:17.492 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:32:17.493 7f3464c9b048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:32:17.493 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:17.493 7f3464c9b048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:32:17.493 7f3464c9b048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:17.493 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:17.493 7f3464c9b048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:32:17.493 7f3464c9b048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:21.492 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:32:21.493 7f3464c9b048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:32:21.493 7f3464c9b048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:32:21.493 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:21.494 7f3464c9b048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:32:21.494 7f3464c9b048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:21.494 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:32:21.494 7f3464c9b048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:32:21.494 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:21.495 7f3464c9b048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:32:21.495 7f3464c9b048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:21.495 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:21.495 7f3464c9b048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:32:21.495 7f3464c9b048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:25.494 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:32:25.496 7f3464c9b048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:32:25.496 7f3464c9b048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:32:25.496 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:25.496 7f3464c9b048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:32:25.496 7f3464c9b048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:25.496 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:32:25.496 7f3464c9b048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:32:25.497 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:25.497 7f3464c9b048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:32:25.497 7f3464c9b048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:25.497 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:25.497 7f3464c9b048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:32:25.497 7f3464c9b048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:29.497 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:32:29.498 7f3464c9b048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:32:29.498 7f3464c9b048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:32:29.498 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:29.498 7f3464c9b048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:32:29.498 7f3464c9b048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:29.498 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:32:29.499 7f3464c9b048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:32:29.499 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:29.499 7f3464c9b048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:32:29.499 7f3464c9b048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:29.499 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:29.499 7f3464c9b048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:32:29.499 7f3464c9b048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:33.500 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:32:33.501 7f3464c9b048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:32:33.501 7f3464c9b048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:32:33.501 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:33.501 7f3464c9b048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:32:33.501 7f3464c9b048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:33.501 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:32:33.501 7f3464c9b048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:32:33.502 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:33.502 7f3464c9b048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:32:33.502 7f3464c9b048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:33.502 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:33.502 7f3464c9b048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:32:33.502 7f3464c9b048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:37.502 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:32:37.504 7f3464c9b048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:32:37.504 7f3464c9b048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:32:37.504 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:37.504 7f3464c9b048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:32:37.504 7f3464c9b048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:37.504 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:32:37.505 7f3464c9b048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:32:37.505 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:37.505 7f3464c9b048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:32:37.505 7f3464c9b048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:37.505 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:37.505 7f3464c9b048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:32:37.505 7f3464c9b048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:41.505 7f3464c9b588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:32:41.506 7f3464c9b588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:32:41.506 7f3464c9b588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:32:41.506 7f3464c9b588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:41.507 7f3464c9b588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:32:41.507 7f3464c9b588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:41.507 7f3464c9b588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:32:41.507 7f3464c9b588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:32:41.507 7f3464c9b588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:41.508 7f3464c9b588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:32:41.508 7f3464c9b588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:41.508 7f3464c9b588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:41.508 7f3464c9b588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:32:41.508 7f3464c9b588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:45.507 7f3464c9b588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:32:45.508 7f3464c9b588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:32:45.508 7f3464c9b588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:32:45.508 7f3464c9b588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:45.508 7f3464c9b588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:32:45.508 7f3464c9b588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:45.508 7f3464c9b588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:32:45.509 7f3464c9b588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:32:45.509 7f3464c9b588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:45.509 7f3464c9b588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:32:45.509 7f3464c9b588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:45.509 7f3464c9b588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:45.509 7f3464c9b588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:32:45.509 7f3464c9b588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:49.509 7f3464c9b588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:32:49.511 7f3464c9b588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:32:49.511 7f3464c9b588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:32:49.511 7f3464c9b588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:49.511 7f3464c9b588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:32:49.511 7f3464c9b588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:49.511 7f3464c9b588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:32:49.512 7f3464c9b588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:32:49.512 7f3464c9b588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:49.512 7f3464c9b588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:32:49.512 7f3464c9b588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:49.512 7f3464c9b588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:49.512 7f3464c9b588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:32:49.512 7f3464c9b588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:53.512 7f3464d73f60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:32:53.513 7f3464d73f60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:32:53.513 7f3464d73f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:32:53.513 7f3464d73f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:53.513 7f3464d73f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:32:53.513 7f3464d73f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:53.513 7f3464d73f60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:32:53.514 7f3464d73f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:32:53.514 7f3464d73f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:53.514 7f3464d73f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:32:53.514 7f3464d73f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:53.514 7f3464d73f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:53.514 7f3464d73f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:32:53.514 7f3464d73f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:57.513 7f3464c9bac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:32:57.514 7f3464c9bac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:32:57.514 7f3464c9bac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:32:57.514 7f3464c9bac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:57.515 7f3464c9bac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:32:57.515 7f3464c9bac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:32:57.515 7f3464c9bac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:32:57.515 7f3464c9bac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:32:57.515 7f3464c9bac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:57.515 7f3464c9bac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:32:57.515 7f3464c9bac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:57.515 7f3464c9bac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:32:57.515 7f3464c9bac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:32:57.515 7f3464c9bac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:01.515 7f3464e5ec18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:33:01.516 7f3464e5ec18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:33:01.517 7f3464e5ec18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:33:01.517 7f3464e5ec18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:01.517 7f3464e5ec18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:33:01.517 7f3464e5ec18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:01.517 7f3464e5ec18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:33:01.518 7f3464e5ec18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:33:01.518 7f3464e5ec18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:01.518 7f3464e5ec18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:33:01.518 7f3464e5ec18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:01.518 7f3464e5ec18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:01.518 7f3464e5ec18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:33:01.518 7f3464e5ec18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:05.517 7f346608f6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:33:05.518 7f346608f6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:33:05.518 7f346608f6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:33:05.518 7f346608f6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:05.518 7f346608f6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:33:05.518 7f346608f6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:05.518 7f346608f6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:33:05.518 7f346608f6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:33:05.519 7f346608f6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:05.519 7f346608f6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:33:05.519 7f346608f6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:05.519 7f346608f6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:05.519 7f346608f6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:33:05.519 7f346608f6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:09.518 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:33:09.519 7f3464c9b048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:33:09.520 7f3464c9b048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:33:09.520 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:09.520 7f3464c9b048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:33:09.520 7f3464c9b048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:09.520 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:33:09.520 7f3464c9b048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:33:09.521 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:09.521 7f3464c9b048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:33:09.521 7f3464c9b048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:09.521 7f3464c9b048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:09.521 7f3464c9b048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:33:09.521 7f3464c9b048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:13.521 7f3468ff86d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:33:13.522 7f3468ff86d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:33:13.522 7f3468ff86d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:33:13.522 7f3468ff86d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:13.523 7f3468ff86d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:33:13.523 7f3468ff86d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:13.523 7f3468ff86d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:33:13.523 7f3468ff86d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:33:13.523 7f3468ff86d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:13.523 7f3468ff86d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:33:13.523 7f3468ff86d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:13.523 7f3468ff86d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:13.523 7f3468ff86d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:33:13.523 7f3468ff86d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:17.523 7f3464c472e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:33:17.524 7f3464c472e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:33:17.524 7f3464c472e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:33:17.524 7f3464c472e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:17.524 7f3464c472e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:33:17.524 7f3464c472e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:17.524 7f3464c472e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:33:17.524 7f3464c472e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:33:17.524 7f3464c472e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:17.525 7f3464c472e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:33:17.525 7f3464c472e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:17.525 7f3464c472e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:17.525 7f3464c472e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:33:17.525 7f3464c472e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:21.525 7f346bdbb048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:33:21.526 7f346bdbb048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:33:21.526 7f346bdbb048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:33:21.526 7f346bdbb048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:21.526 7f346bdbb048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:33:21.526 7f346bdbb048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:21.526 7f346bdbb048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:33:21.526 7f346bdbb048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:33:21.526 7f346bdbb048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:21.526 7f346bdbb048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:33:21.527 7f346bdbb048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:21.527 7f346bdbb048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:21.527 7f346bdbb048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:33:21.527 7f346bdbb048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:21.770 7f346ec83348 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 11/03/2025 05:33:21.770 7f346ec83348 [contrail-device-manager] [WARNING]: initializing cassandra cluster 11/03/2025 05:33:21.770 7f346ec83348 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 11/03/2025 05:33:21.770 7f346bdd7438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:21.771 7f346bdd7438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 11/03/2025 05:33:21.771 7f346bdd7438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:21.771 7f346ec83348 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 11/03/2025 05:33:21.798 7f346ec83348 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 11/03/2025 05:33:21.799 7f3464dc3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:21.799 7f3464dc3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Initializing description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 11/03/2025 05:33:21.799 7f3464dc3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:21.807 7f346ec83348 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 11/03/2025 05:33:21.807 7f3464dc3048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:21.807 7f3464dc3048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 11/03/2025 05:33:21.807 7f3464dc3048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:21.820 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/03/2025 05:33:22.274 7f346ec83348 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace', created 11/03/2025 05:33:22.279 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/03/2025 05:33:22.288 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/03/2025 05:33:23.384 7f346ec83348 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None} 11/03/2025 05:33:23.389 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/03/2025 05:33:23.389 7f346ec83348 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 11/03/2025 05:33:23.468 7f346ec83348 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 11/03/2025 05:33:23.475 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/03/2025 05:33:24.412 7f346ec83348 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table', created 11/03/2025 05:33:24.417 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/03/2025 05:33:24.424 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/03/2025 05:33:25.310 7f346ec83348 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table' fixed with properties {'gc_grace_seconds': 864000} 11/03/2025 05:33:25.314 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/03/2025 05:33:25.320 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/03/2025 05:33:25.527 7f3464dfd780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:33:25.528 7f3464dfd780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:33:25.528 7f3464dfd780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:33:25.528 7f3464dfd780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:25.528 7f3464dfd780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 11/03/2025 05:33:25.529 7f3464dfd780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:25.529 7f3464dfd780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:33:25.529 7f3464dfd780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:33:25.529 7f3464dfd780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:25.529 7f3464dfd780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 11/03/2025 05:33:25.529 7f3464dfd780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:25.529 7f3464dfd780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:25.529 7f3464dfd780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:33:25.529 7f3464dfd780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:26.128 7f346ec83348 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table', created 11/03/2025 05:33:26.139 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/03/2025 05:33:26.154 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/03/2025 05:33:27.450 7f346ec83348 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table' fixed with properties {'gc_grace_seconds': 864000} 11/03/2025 05:33:27.461 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/03/2025 05:33:27.478 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/03/2025 05:33:28.407 7f346ec83348 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table', created 11/03/2025 05:33:28.425 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/03/2025 05:33:28.450 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/03/2025 05:33:29.529 7f3464e0bcc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:33:29.530 7f3464e0bcc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:33:29.530 7f3464e0bcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:33:29.531 7f3464e0bcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:29.531 7f3464e0bcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 11/03/2025 05:33:29.531 7f3464e0bcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:29.531 7f3464e0bcc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:33:29.531 7f3464e0bcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:33:29.531 7f3464e0bcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:29.531 7f3464e0bcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 11/03/2025 05:33:29.531 7f3464e0bcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:29.531 7f3464e0bcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:29.532 7f3464e0bcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:33:29.532 7f3464e0bcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:29.761 7f346ec83348 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table' fixed with properties {'gc_grace_seconds': 864000} 11/03/2025 05:33:29.768 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/03/2025 05:33:29.783 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/03/2025 05:33:30.871 7f346ec83348 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table', created 11/03/2025 05:33:30.881 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/03/2025 05:33:30.893 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/03/2025 05:33:31.373 7f346ec83348 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table' fixed with properties {'gc_grace_seconds': 864000} 11/03/2025 05:33:31.378 7f346ec83348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/03/2025 05:33:31.379 7f3466039b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:31.379 7f3466039b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:33:31.379 7f3466039b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:31.422 7f346ec83348 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.48:5673//', 'pyamqp://guest:guest@10.0.0.49:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 11/03/2025 05:33:31.423 7f346ec83348 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 11/03/2025 05:33:31.423 7f3464e06630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:31.423 7f3464e06630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Initializing description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 11/03/2025 05:33:31.423 7f3464e06630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:31.423 7f3464e06630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:31.424 7f3464e06630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 11/03/2025 05:33:31.424 7f3464e06630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:31.439 7f346ec83348 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 11/03/2025 05:33:31.439 7f3464e10c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:31.439 7f3464e10c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:33:31.439 7f3464e10c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:31.529 7f346ec83348 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 11/03/2025 05:33:31.530 7f3464e10438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:31.530 7f3464e10438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 11/03/2025 05:33:31.530 7f3464e10438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:31.544 7f346ec83348 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 11/03/2025 05:33:31.544 7f3464e3c0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:31.545 7f3464e3c0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:33:31.545 7f3464e3c0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:31.548 7f3464e06588 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 11/03/2025 05:33:31.548 7f3464e03048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:31.548 7f3464e03048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 11/03/2025 05:33:31.548 7f3464e03048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:31.566 7f3464e06588 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 11/03/2025 05:33:31.567 7f3464e03a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:31.567 7f3464e03a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:33:31.567 7f3464e03a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:33.532 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:33:33.533 7f3466f76ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:33:33.533 7f3466f76ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:33:33.533 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:33.534 7f3466f76ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:33:33.534 7f3466f76ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:33.534 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:33:33.534 7f3466f76ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:33:33.534 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:33.534 7f3466f76ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:33:33.534 7f3466f76ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:33.534 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:33.534 7f3466f76ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:33:33.534 7f3466f76ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:37.534 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:33:37.535 7f3466f76ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:33:37.536 7f3466f76ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:33:37.536 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:37.536 7f3466f76ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:33:37.536 7f3466f76ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:37.536 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:33:37.536 7f3466f76ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:33:37.536 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:37.536 7f3466f76ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:33:37.536 7f3466f76ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:37.536 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:37.537 7f3466f76ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:33:37.537 7f3466f76ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:41.537 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:33:41.538 7f3466f76ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:33:41.538 7f3466f76ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:33:41.538 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:41.538 7f3466f76ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:33:41.538 7f3466f76ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:41.538 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:33:41.539 7f3466f76ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:33:41.539 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:41.539 7f3466f76ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:33:41.539 7f3466f76ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:41.539 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:41.539 7f3466f76ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:33:41.539 7f3466f76ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:45.539 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:33:45.540 7f3466f76ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:33:45.540 7f3466f76ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:33:45.540 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:45.540 7f3466f76ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:33:45.540 7f3466f76ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:45.540 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:33:45.541 7f3466f76ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:33:45.541 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:45.541 7f3466f76ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:33:45.541 7f3466f76ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:45.541 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:45.541 7f3466f76ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:33:45.541 7f3466f76ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:49.542 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:33:49.543 7f3466f76ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:33:49.543 7f3466f76ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:33:49.543 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:49.544 7f3466f76ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:33:49.544 7f3466f76ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:49.544 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:33:49.544 7f3466f76ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:33:49.544 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:49.544 7f3466f76ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:33:49.544 7f3466f76ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:49.544 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:49.544 7f3466f76ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:33:49.544 7f3466f76ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:53.544 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:33:53.546 7f3466f76ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:33:53.546 7f3466f76ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:33:53.546 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:53.547 7f3466f76ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:33:53.547 7f3466f76ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:53.547 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:33:53.547 7f3466f76ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:33:53.548 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:53.548 7f3466f76ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:33:53.548 7f3466f76ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:53.548 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:53.548 7f3466f76ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:33:53.548 7f3466f76ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:57.546 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:33:57.547 7f3466f76ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/03/2025 05:33:57.547 7f3466f76ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:33:57.548 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:57.548 7f3466f76ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:33:57.548 7f3466f76ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:33:57.548 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/03/2025 05:33:57.548 7f3466f76ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/03/2025 05:33:57.548 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:57.548 7f3466f76ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/03/2025 05:33:57.548 7f3466f76ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:57.548 7f3466f76ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:57.548 7f3466f76ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1762147897468564 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.49:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/03/2025 05:33:57.548 7f3466f76ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/03/2025 05:33:59.850 7f3464e11780 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4601-2.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.49', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4601-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4601-2.'], 'id_perms': {'created': '2025-11-03T05:33:59.840050', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-03T05:33:59.840050', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13587317455456977819, 'uuid_mslong': 9815787267308931486}}, 'parent_type': 'global-system-config', 'parent_uuid': '6eefe408-2e50-4120-b0d7-10f622ac5330', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '8838ae83-6ac0-499e-bc8f-d93caefa439b'}, 'oper': 'CREATE', 'request-id': 'req-bf86f156-7d8c-4ed3-ae81-3404becb07a2', 'type': 'analytics_node', 'uuid': '8838ae83-6ac0-499e-bc8f-d93caefa439b'} 11/03/2025 05:34:00.293 7f3464e11780 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4601-3.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.50', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4601-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4601-3.'], 'id_perms': {'created': '2025-11-03T05:34:00.282251', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-03T05:34:00.282251', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11847746144642822035, 'uuid_mslong': 10368953596982216898}}, 'parent_type': 'global-system-config', 'parent_uuid': '6eefe408-2e50-4120-b0d7-10f622ac5330', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '8fe5ec5c-00a2-48c2-a46b-a658b4706f93'}, 'oper': 'CREATE', 'request-id': 'req-829bce38-8dff-4064-9985-58cac67b2b44', 'type': 'analytics_node', 'uuid': '8fe5ec5c-00a2-48c2-a46b-a658b4706f93'} 11/03/2025 05:34:00.378 7f3464e11780 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4601-1.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.48', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4601-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4601-1.'], 'id_perms': {'created': '2025-11-03T05:34:00.369893', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-03T05:34:00.369893', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13248047238060851341, 'uuid_mslong': 5686278822121390782}}, 'parent_type': 'global-system-config', 'parent_uuid': '6eefe408-2e50-4120-b0d7-10f622ac5330', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '4ee9b876-265f-42be-b7da-84c59836408d'}, 'oper': 'CREATE', 'request-id': 'req-99ca7e93-8d61-43c9-8e1c-d92da8181972', 'type': 'analytics_node', 'uuid': '4ee9b876-265f-42be-b7da-84c59836408d'} 11/03/2025 05:34:01.548 7f3464e07198 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/03/2025 05:34:01.550 7f3464e07198 [contrail-device-manager] [INFO]: Session Event: TCP Connected 11/03/2025 05:34:01.550 7f3464e07198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/03/2025 05:34:01.550 7f3464e07198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:34:01.550 7f3464e07198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4601-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.48:5673, 10.0.0.49:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.48:9041, 10.0.0.49:9041, 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.48:2181, 10.0.0.49:2181, 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.48,10.0.0.49,10.0.0.50:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/03/2025 05:34:01.550 7f3464e07198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/03/2025 05:34:01.550 7f3464e07198 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect] 11/03/2025 05:34:01.550 7f3464e07198 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19 11/03/2025 05:34:01.551 7f3464e07198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 11/03/2025 05:34:01.551 7f3464e07198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/03/2025 05:34:01.551 7f3464e07198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/03/2025 05:34:01.551 7f3464e07f60 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 11/03/2025 05:34:01.552 7f3464e07f60 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 11/03/2025 05:34:11.935 7f3464e07e10 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001016 11/03/2025 05:34:11.936 7f3464e07e10 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 11/03/2025 05:34:11.937 7f3464e0beb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 11/03/2025 05:34:11.937 7f3464e0beb8 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 0 11/03/2025 05:34:11.938 7f3464e0beb8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 11/03/2025 05:34:11.938 7f3464e0beb8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/03/2025 05:34:11.938 7f3464e0beb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 11/03/2025 05:34:11.939 7f3464d73a20 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 11/03/2025 05:34:11.941 7f3464d73a20 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 11/03/2025 05:34:17.422 7f3464e11780 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4601-2.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.49', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4601-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4601-2.'], 'id_perms': {'created': '2025-11-03T05:34:17.412615', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-03T05:34:17.412615', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11009669563244191897, 'uuid_mslong': 12349889079331606094}}, 'parent_type': 'global-system-config', 'parent_uuid': '6eefe408-2e50-4120-b0d7-10f622ac5330', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'ab639eaf-6509-4e4e-98ca-3422d399a099'}, 'oper': 'CREATE', 'request-id': 'req-e41abc6e-cc3e-48ac-9c5e-8d7e19799c28', 'type': 'analytics_alarm_node', 'uuid': 'ab639eaf-6509-4e4e-98ca-3422d399a099'} 11/03/2025 05:34:18.388 7f3464e11780 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4601-3.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.50', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4601-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4601-3.'], 'id_perms': {'created': '2025-11-03T05:34:18.359322', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-03T05:34:18.359322', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10646807172032095052, 'uuid_mslong': 2419122298861207598}}, 'parent_type': 'global-system-config', 'parent_uuid': '6eefe408-2e50-4120-b0d7-10f622ac5330', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '219272e9-ebd9-402e-93c1-0eb6b90de34c'}, 'oper': 'CREATE', 'request-id': 'req-174e835b-f464-4e51-a0b7-336df4be0cf4', 'type': 'analytics_alarm_node', 'uuid': '219272e9-ebd9-402e-93c1-0eb6b90de34c'} 11/03/2025 05:34:18.519 7f3464e11780 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4601-1.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.48', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4601-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4601-1.'], 'id_perms': {'created': '2025-11-03T05:34:18.502398', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-03T05:34:18.502398', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9856848999815289707, 'uuid_mslong': 13540790126159610826}}, 'parent_type': 'global-system-config', 'parent_uuid': '6eefe408-2e50-4120-b0d7-10f622ac5330', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'bbea8ce0-a7dc-43ca-88ca-8ff106a79b6b'}, 'oper': 'CREATE', 'request-id': 'req-51cbef80-bc99-4bc6-85fa-26b28bf70763', 'type': 'analytics_alarm_node', 'uuid': 'bbea8ce0-a7dc-43ca-88ca-8ff106a79b6b'} 11/03/2025 05:34:22.372 7f3464e07e10 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001296 11/03/2025 05:34:22.372 7f3464e07e10 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 11/03/2025 05:34:22.373 7f3464e0d630 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 11/03/2025 05:34:22.373 7f3464e0d630 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 2 11/03/2025 05:34:22.373 7f3464e0d630 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 11/03/2025 05:34:22.373 7f3464e0d630 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/03/2025 05:34:27.973 7f3464e11780 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4601-3.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.50', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4601-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4601-3.'], 'id_perms': {'created': '2025-11-03T05:34:27.957396', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-03T05:34:27.957396', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11844897932824508851, 'uuid_mslong': 12411757286714655834}}, 'parent_type': 'global-system-config', 'parent_uuid': '6eefe408-2e50-4120-b0d7-10f622ac5330', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'ac3f6b7d-f0fe-445a-a461-87e9b6c379b3'}, 'oper': 'CREATE', 'request-id': 'req-ee326bed-0699-48d2-9751-8af80abd3b1a', 'type': 'analytics_snmp_node', 'uuid': 'ac3f6b7d-f0fe-445a-a461-87e9b6c379b3'} 11/03/2025 05:34:28.040 7f3464e11780 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4601-2.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.49', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4601-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4601-2.'], 'id_perms': {'created': '2025-11-03T05:34:28.027008', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-03T05:34:28.027008', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11973721199060649603, 'uuid_mslong': 4263321560623237849}}, 'parent_type': 'global-system-config', 'parent_uuid': '6eefe408-2e50-4120-b0d7-10f622ac5330', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '3b2a5c5f-1036-4ad9-a62b-33fdb5930683'}, 'oper': 'CREATE', 'request-id': 'req-60eddd32-7230-4059-9cd8-0ce593e3eac9', 'type': 'analytics_snmp_node', 'uuid': '3b2a5c5f-1036-4ad9-a62b-33fdb5930683'} 11/03/2025 05:34:28.113 7f3464e11780 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4601-1.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.48', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4601-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4601-1.'], 'id_perms': {'created': '2025-11-03T05:34:28.095191', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-03T05:34:28.095191', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11434402916536677839, 'uuid_mslong': 5460073038169721640}}, 'parent_type': 'global-system-config', 'parent_uuid': '6eefe408-2e50-4120-b0d7-10f622ac5330', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '4bc6137f-dfcb-4b28-9eaf-28dedadbf5cf'}, 'oper': 'CREATE', 'request-id': 'req-999cc4b1-53e7-431c-9006-b2d14c41432f', 'type': 'analytics_snmp_node', 'uuid': '4bc6137f-dfcb-4b28-9eaf-28dedadbf5cf'} 11/03/2025 05:36:14.236 7f3464e11780 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-f96a8cfc-3763-441a-9f3c-90c7722ba36b', 'type': 'global_system_config', 'uuid': '6eefe408-2e50-4120-b0d7-10f622ac5330'} 11/03/2025 05:36:14.293 7f3464e11780 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-bb182844-3d57-4f79-9f29-018123a4a358', 'type': 'global_system_config', 'uuid': '6eefe408-2e50-4120-b0d7-10f622ac5330'} 11/03/2025 05:36:14.365 7f3464e11780 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4601-2'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-4601-2', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4601-2'], 'id_perms': {'created': '2025-11-03T05:36:14.357941', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-03T05:36:14.357941', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12598456904268505094, 'uuid_mslong': 1045020975459681177}}, 'parent_type': 'global-system-config', 'parent_uuid': '6eefe408-2e50-4120-b0d7-10f622ac5330', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '0e80a90a-591b-4399-aed6-b5ce10fde006', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.193'}, 'oper': 'CREATE', 'request-id': 'req-9848cbcc-d00f-4d6b-8a99-00f7ee5155e1', 'type': 'virtual_router', 'uuid': '0e80a90a-591b-4399-aed6-b5ce10fde006'} 11/03/2025 05:36:14.438 7f3464e11780 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4601-1'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-4601-1', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4601-1'], 'id_perms': {'created': '2025-11-03T05:36:14.430119', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-03T05:36:14.430119', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13083029711985625561, 'uuid_mslong': 12119477546602220944}}, 'parent_type': 'global-system-config', 'parent_uuid': '6eefe408-2e50-4120-b0d7-10f622ac5330', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'a831089d-ec8c-4d90-b590-4234c8c8c9d9', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.25'}, 'oper': 'CREATE', 'request-id': 'req-88c7c99d-cd0b-444c-afab-b5a890d603d4', 'type': 'virtual_router', 'uuid': 'a831089d-ec8c-4d90-b590-4234c8c8c9d9'} 11/03/2025 05:36:14.775 7f3464e11780 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4601-2', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4601-2', 'vhost0'], 'id_perms': {'created': '2025-11-03T05:36:14.730577', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-03T05:36:14.730577', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10375590155082179503, 'uuid_mslong': 6427022095104954003}}, 'parent_type': 'virtual-router', 'parent_uuid': '0e80a90a-591b-4399-aed6-b5ce10fde006', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '59315e8b-2e51-4e93-8ffd-804618b6a3af', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:59:31:5e:8b:2e']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': '068a3b00-fb84-4804-8a93-1fb2ea2d80c8'}]}, 'oper': 'CREATE', 'request-id': 'req-e66c9a11-59bf-4024-abb6-e7a66f0b63ea', 'type': 'virtual_machine_interface', 'uuid': '59315e8b-2e51-4e93-8ffd-804618b6a3af'} 11/03/2025 05:36:14.819 7f3464e11780 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4601-2', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-e66c9a11-59bf-4024-abb6-e7a66f0b63ea', 'type': 'virtual_machine_interface', 'uuid': '59315e8b-2e51-4e93-8ffd-804618b6a3af'} 11/03/2025 05:36:14.858 7f3464e11780 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4601-1', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4601-1', 'vhost0'], 'id_perms': {'created': '2025-11-03T05:36:14.813255', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-11-03T05:36:14.813255', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12262691259942105476, 'uuid_mslong': 858925307796998956}}, 'parent_type': 'virtual-router', 'parent_uuid': 'a831089d-ec8c-4d90-b590-4234c8c8c9d9', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '0beb8404-43d5-4b2c-aa2d-d4bafb22f184', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:0b:eb:84:04:43']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': '068a3b00-fb84-4804-8a93-1fb2ea2d80c8'}]}, 'oper': 'CREATE', 'request-id': 'req-d6bb7975-096e-4d91-b138-7c34da7741a1', 'type': 'virtual_machine_interface', 'uuid': '0beb8404-43d5-4b2c-aa2d-d4bafb22f184'} 11/03/2025 05:36:14.914 7f3464e11780 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4601-1', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-d6bb7975-096e-4d91-b138-7c34da7741a1', 'type': 'virtual_machine_interface', 'uuid': '0beb8404-43d5-4b2c-aa2d-d4bafb22f184'} 11/03/2025 05:36:15.308 7f3464e11780 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-3865df9b-7254-4075-a38e-cd8ff8681247', 'type': 'access_control_list', 'uuid': 'd01eb138-4567-48ac-af88-90ba449a0904'} 11/03/2025 05:36:53.030 7f3464dc7e10 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 11/03/2025 05:37:17.992 7f3464dc7e10 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 11/03/2025 05:39:14.291 7f3464dfacc0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map