03/27/2025 00:47:07.943 7f5aa7300848 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0]
03/27/2025 00:47:07.943 7f5aa7300848 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False]
03/27/2025 00:47:07.944 7f5aa7300848 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False]
03/27/2025 00:47:07.944 7f5a9e93e390 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none]
03/27/2025 00:47:07.944 7f5a9e93e390 [contrail-device-manager] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-2962-3., 1> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 00:47:07.944 7f5a9e93e390 [contrail-device-manager] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0, 1> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
03/27/2025 00:47:07.945 7f5a9e93e390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle]
03/27/2025 00:47:07.945 7f5a9e93e390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:07.945 7f5a9e93e390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
03/27/2025 00:47:07.945 7f5a9e93e390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:07.945 7f5a9e93e390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:07.945 7f5a9e93e390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  start_time = 1743036427944866  collector_name =   collector_ip =   collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 1  >>  
03/27/2025 00:47:07.945 7f5a9e93e390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:07.999 7f5aa7300848 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.36:5673//', 'pyamqp://guest:guest@10.0.0.27:5673//', 'pyamqp://guest:guest@10.0.0.43:5673//']
03/27/2025 00:47:08.009 7f5aa7300848 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down
03/27/2025 00:47:08.010 7f5a9e890630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:08.010 7f5a9e890630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Initializing  description =   >>,  ]  description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down  >>,  ]  >>  
03/27/2025 00:47:08.010 7f5a9e890630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:08.010 7f5a9e890630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:08.010 7f5a9e890630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Down  description =   >>,  ]  description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down  >>,  ]  >>  
03/27/2025 00:47:08.010 7f5a9e890630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:08.018 7f5aa7300848 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED <Connection: amqp://guest:**@10.0.0.36:5673// at 0x7f5a9e8405f8>
03/27/2025 00:47:08.018 7f5a9e8902e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:08.018 7f5a9e8902e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
03/27/2025 00:47:08.018 7f5a9e8902e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:08.068 7f5aa7300848 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down
03/27/2025 00:47:08.069 7f5a9e8908d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:08.069 7f5a9e8908d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Down  description =   >>,  ]  description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down  >>,  ]  >>  
03/27/2025 00:47:08.069 7f5a9e8908d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:08.074 7f5aa7300848 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED <Connection: amqp://guest:**@10.0.0.36:5673// at 0x7f5a9e8b7f98>
03/27/2025 00:47:08.075 7f5aad142978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:08.075 7f5aad142978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
03/27/2025 00:47:08.075 7f5aad142978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:08.102 7f5aa7300848 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run
03/27/2025 00:47:08.102 7f5aa7300848 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql
03/27/2025 00:47:08.102 7f5aa7300848 [contrail-device-manager] [WARNING]: initializing cassandra cluster
03/27/2025 00:47:08.103 7f5aa7300848 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create
03/27/2025 00:47:08.103 7f5aa41c7780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet
03/27/2025 00:47:08.103 7f5aa41c7780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection
03/27/2025 00:47:08.104 7f5aa10dfa20 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet
03/27/2025 00:47:08.104 7f5aa10dfa20 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection
03/27/2025 00:47:08.104 7f5a9e72d240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:08.104 7f5a9e72d240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Initializing  description =   >>,  ]  description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down  >>,  ]  >>  
03/27/2025 00:47:08.104 7f5a9e72d240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:08.105 7f5aa7300848 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created
03/27/2025 00:47:08.110 7f5aa10dfa20 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established <Connection: amqp://guest:**@10.0.0.36:5673// at 0x7f5a9e7323c8>
03/27/2025 00:47:08.110 7f5aa41c7780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established <Connection: amqp://guest:**@10.0.0.36:5673// at 0x7f5a9e7323c8>
03/27/2025 00:47:08.110 7f5aa41c7780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer
03/27/2025 00:47:08.144 7f5aa7300848 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected
03/27/2025 00:47:08.146 7f5a9e6874e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:08.146 7f5a9e6874e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Initializing  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Initializing  description =   >>,  ]  description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down  >>,  ]  >>  
03/27/2025 00:47:08.146 7f5a9e6874e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:08.155 7f5aa7300848 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS
03/27/2025 00:47:08.155 7f5a9e682780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:08.155 7f5a9e682780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Initializing  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>,  ]  description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down  >>,  ]  >>  
03/27/2025 00:47:08.155 7f5a9e682780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:08.166 7f5aa7300848 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready
03/27/2025 00:47:08.197 7f5aa7300848 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready
03/27/2025 00:47:09.093 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change...
03/27/2025 00:47:10.091 7f5aa7300848 [contrail-device-manager] [WARNING]: table 'obj_uuid_table', created
03/27/2025 00:47:10.097 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock released.
03/27/2025 00:47:11.946 7f5a9e625978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:47:11.947 7f5a9e625978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:47:11.947 7f5a9e625978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:47:11.947 7f5a9e625978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:47:11.947 7f5a9e625978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Initializing  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down  >>,  ]  >>  
03/27/2025 00:47:11.947 7f5a9e625978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:47:11.948 7f5a9e625978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:47:11.948 7f5a9e625978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:47:11.948 7f5a9e625978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:11.948 7f5a9e625978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Initializing  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down  >>,  ]  >>  
03/27/2025 00:47:11.948 7f5a9e625978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:11.948 7f5a9e625978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:11.948 7f5a9e625978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.43:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:47:11.948 7f5a9e625978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:12.039 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change...
03/27/2025 00:47:13.162 7f5aa7300848 [contrail-device-manager] [WARNING]: table 'obj_uuid_table' fixed with properties {'gc_grace_seconds': 864000}
03/27/2025 00:47:13.170 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock released.
03/27/2025 00:47:15.260 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change...
03/27/2025 00:47:15.948 7f5a9e626390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:47:15.950 7f5a9e626390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:47:15.950 7f5a9e626390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:47:15.950 7f5a9e626390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:47:15.950 7f5a9e626390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Initializing  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down  >>,  ]  >>  
03/27/2025 00:47:15.950 7f5a9e626390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:47:15.950 7f5a9e626390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:47:15.951 7f5a9e626390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:47:15.951 7f5a9e626390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:15.951 7f5a9e626390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Initializing  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down  >>,  ]  >>  
03/27/2025 00:47:15.952 7f5a9e626390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:15.952 7f5a9e626390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:15.952 7f5a9e626390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.27:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:47:15.952 7f5a9e626390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:16.083 7f5aa7300848 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table', created
03/27/2025 00:47:16.091 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock released.
03/27/2025 00:47:19.020 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change...
03/27/2025 00:47:19.950 7f5a9e626390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:47:19.952 7f5a9e626390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:47:19.953 7f5a9e626390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:47:19.953 7f5a9e626390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:47:19.953 7f5a9e626390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Initializing  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down  >>,  ]  >>  
03/27/2025 00:47:19.954 7f5a9e626390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:47:19.954 7f5a9e626390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:47:19.954 7f5a9e626390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:47:19.954 7f5a9e626390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:19.955 7f5a9e626390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Initializing  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down  >>,  ]  >>  
03/27/2025 00:47:19.955 7f5a9e626390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:19.955 7f5a9e626390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:19.955 7f5a9e626390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.36:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:47:19.955 7f5a9e626390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:20.139 7f5aa7300848 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table' fixed with properties {'gc_grace_seconds': 864000}
03/27/2025 00:47:20.145 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock released.
03/27/2025 00:47:22.911 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change...
03/27/2025 00:47:22.914 7f5aa7300848 [contrail-device-manager] [WARNING]: table 'obj_shared_table', created
03/27/2025 00:47:22.920 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock released.
03/27/2025 00:47:23.953 7f5a9e626eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:47:23.954 7f5a9e626eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:47:23.954 7f5a9e626eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:47:23.954 7f5a9e626eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:47:23.954 7f5a9e626eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Initializing  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down  >>,  ]  >>  
03/27/2025 00:47:23.954 7f5a9e626eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:47:23.954 7f5a9e626eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:47:23.955 7f5a9e626eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:47:23.955 7f5a9e626eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:23.955 7f5a9e626eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Initializing  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down  >>,  ]  >>  
03/27/2025 00:47:23.955 7f5a9e626eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:23.955 7f5a9e626eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:23.955 7f5a9e626eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.43:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:47:23.955 7f5a9e626eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:25.187 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change...
03/27/2025 00:47:26.112 7f5aa7300848 [contrail-device-manager] [WARNING]: table 'obj_shared_table' fixed with properties {'gc_grace_seconds': 864000}
03/27/2025 00:47:26.119 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock released.
03/27/2025 00:47:26.119 7f5aa3f170f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:26.119 7f5aa3f170f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:47:26.120 7f5aa3f170f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:26.134 7f5aa7300848 [contrail-device-manager] [ERROR]: Error in db walk read 'type'
03/27/2025 00:47:26.135 7f5a9dda1780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:26.135 7f5a9dda1780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Initializing  description =   >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Zookeeper:Zookeeper[] connection down  >>,  ]  >>  
03/27/2025 00:47:26.135 7f5a9dda1780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:26.149 7f5a9e61ccc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:26.149 7f5a9e61ccc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:47:26.149 7f5a9e61ccc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:26.150 7f5aa7300848 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy
03/27/2025 00:47:26.150 7f5aa7300848 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer
03/27/2025 00:47:26.150 7f5aa41c7780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection
03/27/2025 00:47:26.150 7f5aa7300848 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2962-3.
03/27/2025 00:47:26.157 7f5aa41c7780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established <Connection: amqp://guest:**@10.0.0.36:5673// at 0x7f5a9e7323c8>
03/27/2025 00:47:26.351 7f5aa41c7780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2962-3.']
03/27/2025 00:47:26.397 7f5aa41c7780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection
03/27/2025 00:47:26.405 7f5aa41c7780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established <Connection: amqp://guest:**@10.0.0.36:5673// at 0x7f5a9e7323c8>
03/27/2025 00:47:26.523 7f5aa41c7780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2962-3.']
03/27/2025 00:47:26.661 7f5aa7300848 [contrail-device-manager] [DEBUG]: Fetching all containers
03/27/2025 00:47:26.705 7f5aa7300848 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1
03/27/2025 00:47:26.884 7f5aa7300848 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2962-3..config_queue
03/27/2025 00:47:26.884 7f5aa7300848 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2962-3..tftp_queue
03/27/2025 00:47:27.385 7f5aa7300848 [contrail-device-manager] [WARNING]: Process 140 prepared to run in Full mode...
03/27/2025 00:47:27.385 7f5aa7300848 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 
03/27/2025 00:47:27.385 7f5aa7300848 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096
03/27/2025 00:47:27.388 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page
03/27/2025 00:47:27.388 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq]
03/27/2025 00:47:27.389 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page
03/27/2025 00:47:27.389 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq]
03/27/2025 00:47:27.390 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page
03/27/2025 00:47:27.390 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq]
03/27/2025 00:47:27.394 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page
03/27/2025 00:47:27.394 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList]
03/27/2025 00:47:27.394 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList]
03/27/2025 00:47:27.394 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList]
03/27/2025 00:47:27.394 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList]
03/27/2025 00:47:27.394 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList]
03/27/2025 00:47:27.394 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList]
03/27/2025 00:47:27.394 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList]
03/27/2025 00:47:27.395 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList]
03/27/2025 00:47:27.395 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList]
03/27/2025 00:47:27.395 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList]
03/27/2025 00:47:27.395 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList]
03/27/2025 00:47:27.395 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList]
03/27/2025 00:47:27.396 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page
03/27/2025 00:47:27.396 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest]
03/27/2025 00:47:27.397 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest]
03/27/2025 00:47:27.397 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest]
03/27/2025 00:47:27.397 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page
03/27/2025 00:47:27.397 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer]
03/27/2025 00:47:27.397 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient]
03/27/2025 00:47:27.398 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page
03/27/2025 00:47:27.398 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest]
03/27/2025 00:47:27.398 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest]
03/27/2025 00:47:27.398 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq]
03/27/2025 00:47:27.398 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq]
03/27/2025 00:47:27.398 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq]
03/27/2025 00:47:27.399 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page
03/27/2025 00:47:27.399 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq]
03/27/2025 00:47:27.399 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet]
03/27/2025 00:47:27.399 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus]
03/27/2025 00:47:27.399 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet]
03/27/2025 00:47:27.399 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus]
03/27/2025 00:47:27.399 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet]
03/27/2025 00:47:27.399 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset]
03/27/2025 00:47:27.399 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus]
03/27/2025 00:47:27.399 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet]
03/27/2025 00:47:27.399 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus]
03/27/2025 00:47:27.399 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest]
03/27/2025 00:47:27.399 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq]
03/27/2025 00:47:27.400 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq]
03/27/2025 00:47:27.400 7f5aa7300848 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq]
03/27/2025 00:47:27.402 7f5aa7300848 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue
03/27/2025 00:47:27.415 7f5a9dd50b70 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port
03/27/2025 00:47:27.416 7f5a9dd50b70 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096
03/27/2025 00:47:27.470 7f5a9d50ed68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:27.470 7f5a9d50ed68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Initializing  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down  >>,  ]  >>  
03/27/2025 00:47:27.470 7f5a9d50ed68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:27.484 7f5a9d46fc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:27.484 7f5a9d46fc18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Down  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down  >>,  ]  >>  
03/27/2025 00:47:27.484 7f5a9d46fc18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:27.549 7f5aa41c7780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection
03/27/2025 00:47:27.556 7f5aa41c7780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established <Connection: amqp://guest:**@10.0.0.36:5673// at 0x7f5a9e7323c8>
03/27/2025 00:47:27.942 7f5aa41c7780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2962-3.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2962-3..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2962-3..tftp_queue', 'device_manager_ztp.ztp_queue']
03/27/2025 00:47:27.954 7f5a9d3fcd68 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:47:27.956 7f5a9d3fcd68 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:47:27.956 7f5a9d3fcd68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:47:27.956 7f5a9d3fcd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:47:27.956 7f5a9d3fcd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Down  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down  >>,  ]  >>  
03/27/2025 00:47:27.956 7f5a9d3fcd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:47:27.957 7f5a9d3fcd68 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:47:27.957 7f5a9d3fcd68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:47:27.957 7f5a9d3fcd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:27.957 7f5a9d3fcd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Down  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down  >>,  ]  >>  
03/27/2025 00:47:27.957 7f5a9d3fcd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:27.957 7f5a9d3fcd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:27.957 7f5a9d3fcd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.27:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:47:27.957 7f5a9d3fcd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:31.957 7f5a9d3b12e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:47:31.959 7f5a9d3b12e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:47:31.959 7f5a9d3b12e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:47:31.959 7f5a9d3b12e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:47:31.959 7f5a9d3b12e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Down  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down  >>,  ]  >>  
03/27/2025 00:47:31.959 7f5a9d3b12e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:47:31.959 7f5a9d3b12e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:47:31.960 7f5a9d3b12e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:47:31.960 7f5a9d3b12e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:31.960 7f5a9d3b12e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Down  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down  >>,  ]  >>  
03/27/2025 00:47:31.960 7f5a9d3b12e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:31.960 7f5a9d3b12e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:31.960 7f5a9d3b12e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.36:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:47:31.960 7f5a9d3b12e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:35.959 7f5a9d375c18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:47:35.960 7f5a9d375c18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:47:35.960 7f5a9d375c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:47:35.960 7f5a9d375c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:47:35.960 7f5a9d375c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Down  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down  >>,  ]  >>  
03/27/2025 00:47:35.961 7f5a9d375c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:47:35.961 7f5a9d375c18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:47:35.961 7f5a9d375c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:47:35.961 7f5a9d375c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:35.961 7f5a9d375c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Down  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down  >>,  ]  >>  
03/27/2025 00:47:35.961 7f5a9d375c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:35.961 7f5a9d375c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:35.961 7f5a9d375c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.43:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:47:35.961 7f5a9d375c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:36.666 7f5a9d382ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:36.666 7f5a9d382ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:47:36.666 7f5a9d382ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:39.961 7f5a9d24a4e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:47:39.962 7f5a9d24a4e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:47:39.963 7f5a9d24a4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:47:39.963 7f5a9d24a4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:47:39.963 7f5a9d24a4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:47:39.963 7f5a9d24a4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:47:39.963 7f5a9d24a4e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:47:39.963 7f5a9d24a4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:47:39.963 7f5a9d24a4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:39.963 7f5a9d24a4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:47:39.963 7f5a9d24a4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:39.963 7f5a9d24a4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:39.964 7f5a9d24a4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.27:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:47:39.964 7f5a9d24a4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:43.963 7f5a9d24a780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:47:43.965 7f5a9d24a780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:47:43.965 7f5a9d24a780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:47:43.965 7f5a9d24a780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:47:43.965 7f5a9d24a780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:47:43.966 7f5a9d24a780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:47:43.966 7f5a9d24a780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:47:43.966 7f5a9d24a780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:47:43.966 7f5a9d24a780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:43.966 7f5a9d24a780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:47:43.966 7f5a9d24a780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:43.966 7f5a9d24a780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:43.967 7f5a9d24a780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.36:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:47:43.967 7f5a9d24a780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:47.965 7f5a9d375198 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:47:47.966 7f5a9d375198 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:47:47.966 7f5a9d375198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:47:47.966 7f5a9d375198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:47:47.967 7f5a9d375198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:47:47.967 7f5a9d375198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:47:47.967 7f5a9d375198 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:47:47.967 7f5a9d375198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:47:47.967 7f5a9d375198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:47.967 7f5a9d375198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:47:47.967 7f5a9d375198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:47.967 7f5a9d375198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:47.967 7f5a9d375198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.43:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:47:47.968 7f5a9d375198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:51.967 7f5a9d375e10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:47:51.969 7f5a9d375e10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:47:51.969 7f5a9d375e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:47:51.969 7f5a9d375e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:47:51.969 7f5a9d375e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:47:51.969 7f5a9d375e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:47:51.969 7f5a9d375e10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:47:51.969 7f5a9d375e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:47:51.970 7f5a9d375e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:51.970 7f5a9d375e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:47:51.970 7f5a9d375e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:51.970 7f5a9d375e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:51.970 7f5a9d375e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.27:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:47:51.970 7f5a9d375e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:55.970 7f5a9e61c0f0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:47:55.971 7f5a9e61c0f0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:47:55.972 7f5a9e61c0f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:47:55.972 7f5a9e61c0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:47:55.972 7f5a9e61c0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:47:55.972 7f5a9e61c0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:47:55.972 7f5a9e61c0f0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:47:55.972 7f5a9e61c0f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:47:55.972 7f5a9e61c0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:55.972 7f5a9e61c0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:47:55.972 7f5a9e61c0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:55.972 7f5a9e61c0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:55.973 7f5a9e61c0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.36:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:47:55.973 7f5a9e61c0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:59.974 7f5a9e61c0f0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:47:59.975 7f5a9e61c0f0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:47:59.975 7f5a9e61c0f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:47:59.975 7f5a9e61c0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:47:59.976 7f5a9e61c0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:47:59.976 7f5a9e61c0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:47:59.976 7f5a9e61c0f0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:47:59.976 7f5a9e61c0f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:47:59.976 7f5a9e61c0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:59.976 7f5a9e61c0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:47:59.976 7f5a9e61c0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:59.976 7f5a9e61c0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:47:59.977 7f5a9e61c0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.43:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:47:59.977 7f5a9e61c0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:03.977 7f5a9e60d438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:48:03.978 7f5a9e60d438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:48:03.979 7f5a9e60d438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:48:03.979 7f5a9e60d438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:03.979 7f5a9e60d438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:48:03.979 7f5a9e60d438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:03.979 7f5a9e60d438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:48:03.980 7f5a9e60d438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:48:03.980 7f5a9e60d438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:03.980 7f5a9e60d438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:48:03.980 7f5a9e60d438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:03.980 7f5a9e60d438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:03.980 7f5a9e60d438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.27:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:48:03.980 7f5a9e60d438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:07.979 7f5a9d375a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:48:07.980 7f5a9d375a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:48:07.980 7f5a9d375a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:48:07.981 7f5a9d375a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:07.981 7f5a9d375a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:48:07.981 7f5a9d375a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:07.981 7f5a9d375a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:48:07.981 7f5a9d375a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:48:07.982 7f5a9d375a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:07.982 7f5a9d375a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:48:07.982 7f5a9d375a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:07.982 7f5a9d375a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:07.982 7f5a9d375a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.36:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:48:07.982 7f5a9d375a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:11.981 7f5a9d382780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:48:11.983 7f5a9d382780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:48:11.983 7f5a9d382780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:48:11.983 7f5a9d382780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:11.983 7f5a9d382780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:48:11.983 7f5a9d382780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:11.983 7f5a9d382780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:48:11.983 7f5a9d382780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:48:11.983 7f5a9d382780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:11.984 7f5a9d382780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:48:11.984 7f5a9d382780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:11.984 7f5a9d382780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:11.984 7f5a9d382780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.43:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:48:11.984 7f5a9d382780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:15.983 7f5a9d375588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:48:15.984 7f5a9d375588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:48:15.985 7f5a9d375588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:48:15.985 7f5a9d375588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:15.985 7f5a9d375588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:48:15.985 7f5a9d375588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:15.985 7f5a9d375588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:48:15.985 7f5a9d375588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:48:15.985 7f5a9d375588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:15.986 7f5a9d375588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:48:15.986 7f5a9d375588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:15.986 7f5a9d375588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:15.986 7f5a9d375588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.27:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:48:15.986 7f5a9d375588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:19.985 7f5a9d384e10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:48:19.987 7f5a9d384e10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:48:19.987 7f5a9d384e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:48:19.987 7f5a9d384e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:19.988 7f5a9d384e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:48:19.988 7f5a9d384e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:19.988 7f5a9d384e10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:48:19.988 7f5a9d384e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:48:19.988 7f5a9d384e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:19.988 7f5a9d384e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:48:19.988 7f5a9d384e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:19.989 7f5a9d384e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:19.989 7f5a9d384e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.36:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:48:19.989 7f5a9d384e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:22.027 7f5aa7300848 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql
03/27/2025 00:48:22.027 7f5aa7300848 [contrail-device-manager] [WARNING]: initializing cassandra cluster
03/27/2025 00:48:22.028 7f5aa7300848 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create
03/27/2025 00:48:22.028 7f5a9e61c2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:22.028 7f5a9e61c2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Initializing  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down  >>,  ]  >>  
03/27/2025 00:48:22.028 7f5a9e61c2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:22.029 7f5aa7300848 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created
03/27/2025 00:48:22.054 7f5aa7300848 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected
03/27/2025 00:48:22.056 7f5a9d430c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:22.056 7f5a9d430c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Initializing  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Initializing  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[], Zookeeper:Zookeeper[] connection down  >>,  ]  >>  
03/27/2025 00:48:22.056 7f5a9d430c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:22.065 7f5aa7300848 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS
03/27/2025 00:48:22.065 7f5a9d430a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:22.066 7f5a9d430a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Initializing  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down  >>,  ]  >>  
03/27/2025 00:48:22.066 7f5a9d430a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:22.106 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change...
03/27/2025 00:48:23.988 7f5a9d4c2ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:48:23.989 7f5a9d4c2ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:48:23.989 7f5a9d4c2ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:48:23.989 7f5a9d4c2ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:23.989 7f5a9d4c2ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Initializing  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down  >>,  ]  >>  
03/27/2025 00:48:23.989 7f5a9d4c2ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:23.990 7f5a9d4c2ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:48:23.990 7f5a9d4c2ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:48:23.990 7f5a9d4c2ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:23.990 7f5a9d4c2ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Initializing  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down  >>,  ]  >>  
03/27/2025 00:48:23.990 7f5a9d4c2ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:23.990 7f5a9d4c2ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:23.990 7f5a9d4c2ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.43:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:48:23.990 7f5a9d4c2ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:24.004 7f5aa7300848 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace', created
03/27/2025 00:48:24.011 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock released.
03/27/2025 00:48:24.019 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change...
03/27/2025 00:48:24.274 7f5aa7300848 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None}
03/27/2025 00:48:24.281 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock released.
03/27/2025 00:48:24.281 7f5aa7300848 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready
03/27/2025 00:48:24.365 7f5aa7300848 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready
03/27/2025 00:48:24.384 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change...
03/27/2025 00:48:25.167 7f5aa7300848 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table', created
03/27/2025 00:48:25.173 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock released.
03/27/2025 00:48:25.182 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change...
03/27/2025 00:48:26.157 7f5aa7300848 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table' fixed with properties {'gc_grace_seconds': 864000}
03/27/2025 00:48:26.162 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock released.
03/27/2025 00:48:26.171 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change...
03/27/2025 00:48:27.919 7f5aa7300848 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table', created
03/27/2025 00:48:27.927 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock released.
03/27/2025 00:48:27.936 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change...
03/27/2025 00:48:27.989 7f5a9d4dd198 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:48:27.991 7f5a9d4dd198 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:48:27.992 7f5a9d4dd198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:48:27.992 7f5a9d4dd198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:27.992 7f5a9d4dd198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Initializing  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down  >>,  ]  >>  
03/27/2025 00:48:27.992 7f5a9d4dd198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:27.992 7f5a9d4dd198 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:48:27.993 7f5a9d4dd198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:48:27.993 7f5a9d4dd198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:27.993 7f5a9d4dd198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Initializing  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down  >>,  ]  >>  
03/27/2025 00:48:27.993 7f5a9d4dd198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:27.993 7f5a9d4dd198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:27.993 7f5a9d4dd198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.27:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:48:27.993 7f5a9d4dd198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:28.239 7f5aa7300848 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table' fixed with properties {'gc_grace_seconds': 864000}
03/27/2025 00:48:28.245 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock released.
03/27/2025 00:48:28.254 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change...
03/27/2025 00:48:29.093 7f5aa7300848 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table', created
03/27/2025 00:48:29.102 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock released.
03/27/2025 00:48:29.112 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change...
03/27/2025 00:48:31.042 7f5aa7300848 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table' fixed with properties {'gc_grace_seconds': 864000}
03/27/2025 00:48:31.048 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock released.
03/27/2025 00:48:31.057 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change...
03/27/2025 00:48:31.992 7f5a9d274cc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:48:31.993 7f5a9d274cc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:48:31.993 7f5a9d274cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:48:31.994 7f5a9d274cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:31.994 7f5a9d274cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Initializing  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down  >>,  ]  >>  
03/27/2025 00:48:31.994 7f5a9d274cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:31.994 7f5a9d274cc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:48:31.994 7f5a9d274cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:48:31.994 7f5a9d274cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:31.994 7f5a9d274cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Initializing  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down  >>,  ]  >>  
03/27/2025 00:48:31.994 7f5a9d274cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:31.994 7f5a9d274cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:31.995 7f5a9d274cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.36:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:48:31.995 7f5a9d274cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:32.233 7f5aa7300848 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table', created
03/27/2025 00:48:32.238 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock released.
03/27/2025 00:48:32.247 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change...
03/27/2025 00:48:33.191 7f5aa7300848 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table' fixed with properties {'gc_grace_seconds': 864000}
03/27/2025 00:48:33.200 7f5aa7300848 [contrail-device-manager] [WARNING]: Zookeeper lock released.
03/27/2025 00:48:33.201 7f5a9d4d4390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:33.201 7f5a9d4d4390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:48:33.201 7f5a9d4d4390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:33.280 7f5aa7300848 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.36:5673//', 'pyamqp://guest:guest@10.0.0.27:5673//', 'pyamqp://guest:guest@10.0.0.43:5673//']
03/27/2025 00:48:33.280 7f5aa7300848 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down
03/27/2025 00:48:33.281 7f5a9d29a780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:33.281 7f5a9d29a780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Initializing  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down  >>,  ]  >>  
03/27/2025 00:48:33.281 7f5a9d29a780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:33.281 7f5a9d29a780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:33.281 7f5a9d29a780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Down  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down  >>,  ]  >>  
03/27/2025 00:48:33.281 7f5a9d29a780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:33.290 7f5aa7300848 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED <Connection: amqp://guest:**@10.0.0.36:5673// at 0x7f5a9e72ed68>
03/27/2025 00:48:33.290 7f5a9d375198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:33.290 7f5a9d375198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:48:33.290 7f5a9d375198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:33.383 7f5aa7300848 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down
03/27/2025 00:48:33.384 7f5a9d2782e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:33.384 7f5a9d2782e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Down  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down  >>,  ]  >>  
03/27/2025 00:48:33.384 7f5a9d2782e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:33.393 7f5aa7300848 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED <Connection: amqp://guest:**@10.0.0.36:5673// at 0x7f5a9d33e4a8>
03/27/2025 00:48:33.394 7f5a9d2a7f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:33.394 7f5a9d2a7f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:48:33.394 7f5a9d2a7f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:33.398 7f5a9d33c588 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down
03/27/2025 00:48:33.400 7f5a9d49f048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:33.400 7f5a9d49f048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Down  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down  >>,  ]  >>  
03/27/2025 00:48:33.400 7f5a9d49f048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:33.425 7f5a9d33c588 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED <Connection: amqp://guest:**@10.0.0.36:5673// at 0x7f5a9d33e4a8>
03/27/2025 00:48:33.425 7f5a9d33ce10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:33.426 7f5a9d33ce10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:48:33.426 7f5a9d33ce10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:35.995 7f5a9d29ff60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:48:35.997 7f5a9d29ff60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:48:35.997 7f5a9d29ff60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:48:35.997 7f5a9d29ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:35.998 7f5a9d29ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:48:35.998 7f5a9d29ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:35.998 7f5a9d29ff60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:48:35.999 7f5a9d29ff60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:48:35.999 7f5a9d29ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:35.999 7f5a9d29ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:48:35.999 7f5a9d29ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:35.999 7f5a9d29ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:35.999 7f5a9d29ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.43:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:48:35.999 7f5a9d29ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:39.998 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:48:40.000 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:48:40.000 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:48:40.000 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:40.000 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:48:40.001 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:40.001 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:48:40.001 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:48:40.001 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:40.001 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:48:40.001 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:40.001 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:40.001 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.27:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:48:40.001 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:44.001 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:48:44.002 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:48:44.003 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:48:44.003 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:44.003 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:48:44.003 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:44.003 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:48:44.003 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:48:44.003 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:44.004 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:48:44.004 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:44.004 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:44.004 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.36:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:48:44.004 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:48.004 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:48:48.006 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:48:48.006 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:48:48.006 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:48.007 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:48:48.007 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:48.007 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:48:48.007 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:48:48.007 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:48.008 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:48:48.008 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:48.008 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:48.008 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.43:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:48:48.008 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:52.007 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:48:52.010 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:48:52.010 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:48:52.010 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:52.011 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:48:52.011 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:52.011 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:48:52.011 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:48:52.011 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:52.012 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:48:52.012 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:52.012 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:52.012 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.27:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:48:52.012 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:56.011 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:48:56.013 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:48:56.014 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:48:56.014 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:56.014 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:48:56.014 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:48:56.014 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:48:56.014 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:48:56.015 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:56.015 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:48:56.015 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:56.015 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:48:56.015 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.36:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:48:56.015 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:00.014 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:49:00.015 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:49:00.015 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:49:00.016 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:00.016 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:49:00.016 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:00.016 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:49:00.016 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:49:00.016 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:00.016 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:49:00.016 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:00.017 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:00.017 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.43:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:49:00.017 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:04.016 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:49:04.018 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:49:04.019 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:49:04.019 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:04.019 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:49:04.019 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:04.019 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:49:04.019 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:49:04.019 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:04.019 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:49:04.020 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:04.020 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:04.020 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.27:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:49:04.020 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:08.020 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:49:08.022 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:49:08.022 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:49:08.022 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:08.022 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:49:08.022 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:08.022 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:49:08.023 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:49:08.023 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:08.023 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:49:08.023 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:08.023 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:08.024 7f5a9e62fcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.36:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:49:08.024 7f5a9e62fcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:10.153 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain',
             'default-project',
             'ip-fabric',
             '__default__',
             'cn-jenkins-deploy-platform-ansible-os-2962-2'],
 'obj_dict': {'bgp_router_parameters': {'address': '10.20.0.6',
                                        'address_families': {'family': ['route-target',
                                                                        'inet-vpn',
                                                                        'e-vpn',
                                                                        'erm-vpn',
                                                                        'inet6-vpn']},
                                        'admin_down': False,
                                        'auth_data': None,
                                        'autonomous_system': 64512,
                                        'cluster_id': None,
                                        'gateway_address': None,
                                        'hold_time': 0,
                                        'identifier': '10.20.0.6',
                                        'ipv6_gateway_address': None,
                                        'local_autonomous_system': None,
                                        'port': 179,
                                        'router_type': 'control-node',
                                        'source_port': None,
                                        'vendor': 'contrail'},
              'display_name': 'cn-jenkins-deploy-platform-ansible-os-2962-2',
              'fq_name': ['default-domain',
                          'default-project',
                          'ip-fabric',
                          '__default__',
                          'cn-jenkins-deploy-platform-ansible-os-2962-2'],
              'id_perms': {'created': '2025-03-27T00:49:10.118790',
                           'creator': None,
                           'description': None,
                           'enable': True,
                           'last_modified': '2025-03-27T00:49:10.118790',
                           'permissions': {'group': 'member',
                                           'group_access': 7,
                                           'other_access': 7,
                                           'owner': 'admin',
                                           'owner_access': 7},
                           'user_visible': True,
                           'uuid': {'uuid_lslong': 12425158317070077875,
                                    'uuid_mslong': 16902641731411526102}},
              'parent_type': 'routing-instance',
              'parent_uuid': '79bb824d-0424-4147-bb4f-134d718913a2',
              'perms2': {'global_access': 0,
                         'owner': 'cloud-admin',
                         'owner_access': 7,
                         'share': []},
              'uuid': 'ea923f19-c67a-41d6-ac6f-07a876f3cfb3'},
 'oper': 'CREATE',
 'request-id': 'req-52761b31-2f06-468e-8e08-7177efae014d',
 'type': 'bgp_router',
 'uuid': 'ea923f19-c67a-41d6-ac6f-07a876f3cfb3'}
03/27/2025 00:49:10.291 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain',
             'default-project',
             'ip-fabric',
             '__default__',
             'cn-jenkins-deploy-platform-ansible-os-2962-1'],
 'obj_dict': {'bgp_router_parameters': {'address': '10.20.0.14',
                                        'address_families': {'family': ['route-target',
                                                                        'inet-vpn',
                                                                        'e-vpn',
                                                                        'erm-vpn',
                                                                        'inet6-vpn']},
                                        'admin_down': False,
                                        'auth_data': None,
                                        'autonomous_system': 64512,
                                        'cluster_id': None,
                                        'gateway_address': None,
                                        'hold_time': 0,
                                        'identifier': '10.20.0.14',
                                        'ipv6_gateway_address': None,
                                        'local_autonomous_system': None,
                                        'port': 179,
                                        'router_type': 'control-node',
                                        'source_port': None,
                                        'vendor': 'contrail'},
              'display_name': 'cn-jenkins-deploy-platform-ansible-os-2962-1',
              'fq_name': ['default-domain',
                          'default-project',
                          'ip-fabric',
                          '__default__',
                          'cn-jenkins-deploy-platform-ansible-os-2962-1'],
              'id_perms': {'created': '2025-03-27T00:49:10.270500',
                           'creator': None,
                           'description': None,
                           'enable': True,
                           'last_modified': '2025-03-27T00:49:10.270500',
                           'permissions': {'group': 'member',
                                           'group_access': 7,
                                           'other_access': 7,
                                           'owner': 'admin',
                                           'owner_access': 7},
                           'user_visible': True,
                           'uuid': {'uuid_lslong': 13260462220218123851,
                                    'uuid_mslong': 7619521536137383310}},
              'parent_type': 'routing-instance',
              'parent_uuid': '79bb824d-0424-4147-bb4f-134d718913a2',
              'perms2': {'global_access': 0,
                         'owner': 'cloud-admin',
                         'owner_access': 7,
                         'share': []},
              'uuid': '69bdfa77-97e8-4d8e-b806-a021e196c64b'},
 'oper': 'CREATE',
 'request-id': 'req-9cdcae60-8e5f-4e9c-b102-a20de358cb46',
 'type': 'bgp_router',
 'uuid': '69bdfa77-97e8-4d8e-b806-a021e196c64b'}
03/27/2025 00:49:10.352 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain',
             'default-project',
             'ip-fabric',
             '__default__',
             'cn-jenkins-deploy-platform-ansible-os-2962-3'],
 'obj_dict': {'bgp_router_parameters': {'address': '10.20.0.10',
                                        'address_families': {'family': ['route-target',
                                                                        'inet-vpn',
                                                                        'e-vpn',
                                                                        'erm-vpn',
                                                                        'inet6-vpn']},
                                        'admin_down': False,
                                        'auth_data': None,
                                        'autonomous_system': 64512,
                                        'cluster_id': None,
                                        'gateway_address': None,
                                        'hold_time': 0,
                                        'identifier': '10.20.0.10',
                                        'ipv6_gateway_address': None,
                                        'local_autonomous_system': None,
                                        'port': 179,
                                        'router_type': 'control-node',
                                        'source_port': None,
                                        'vendor': 'contrail'},
              'display_name': 'cn-jenkins-deploy-platform-ansible-os-2962-3',
              'fq_name': ['default-domain',
                          'default-project',
                          'ip-fabric',
                          '__default__',
                          'cn-jenkins-deploy-platform-ansible-os-2962-3'],
              'id_perms': {'created': '2025-03-27T00:49:10.325509',
                           'creator': None,
                           'description': None,
                           'enable': True,
                           'last_modified': '2025-03-27T00:49:10.325509',
                           'permissions': {'group': 'member',
                                           'group_access': 7,
                                           'other_access': 7,
                                           'owner': 'admin',
                                           'owner_access': 7},
                           'user_visible': True,
                           'uuid': {'uuid_lslong': 10091530835594032609,
                                    'uuid_mslong': 14515237035480403067}},
              'parent_type': 'routing-instance',
              'parent_uuid': '79bb824d-0424-4147-bb4f-134d718913a2',
              'perms2': {'global_access': 0,
                         'owner': 'cloud-admin',
                         'owner_access': 7,
                         'share': []},
              'uuid': 'c9707b2d-c267-4c7b-8c0c-51d3e77bb5e1'},
 'oper': 'CREATE',
 'request-id': 'req-31c157b4-8323-4818-b487-d684680b2275',
 'type': 'bgp_router',
 'uuid': 'c9707b2d-c267-4c7b-8c0c-51d3e77bb5e1'}
03/27/2025 00:49:10.468 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'],
 'oper': 'UPDATE',
 'request-id': 'req-0adc48af-b7b3-43d2-aa72-7dd6d98cc644',
 'type': 'global_system_config',
 'uuid': 'd46c1ff2-1fd9-4f91-afa4-a8c9ef3fdaac'}
03/27/2025 00:49:10.660 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'],
 'oper': 'UPDATE',
 'request-id': 'req-87b17bad-fc65-485e-9f75-9d586418d13c',
 'type': 'global_system_config',
 'uuid': 'd46c1ff2-1fd9-4f91-afa4-a8c9ef3fdaac'}
03/27/2025 00:49:10.685 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'],
 'oper': 'UPDATE',
 'request-id': 'req-fd01fa1a-c0e5-48d2-9a20-b000d93f22d5',
 'type': 'global_system_config',
 'uuid': 'd46c1ff2-1fd9-4f91-afa4-a8c9ef3fdaac'}
03/27/2025 00:49:10.851 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain',
             'default-project',
             'ip-fabric',
             '__default__',
             'cn-jenkins-deploy-platform-ansible-os-2962-1'],
 'oper': 'UPDATE',
 'request-id': 'req-acda6294-a25e-4baa-b21d-ed719d2529ab',
 'type': 'bgp_router',
 'uuid': '69bdfa77-97e8-4d8e-b806-a021e196c64b'}
03/27/2025 00:49:10.858 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain',
             'default-project',
             'ip-fabric',
             '__default__',
             'cn-jenkins-deploy-platform-ansible-os-2962-2'],
 'oper': 'UPDATE-IMPLICIT',
 'request-id': 'req-acda6294-a25e-4baa-b21d-ed719d2529ab',
 'type': 'bgp_router',
 'uuid': 'ea923f19-c67a-41d6-ac6f-07a876f3cfb3'}
03/27/2025 00:49:10.962 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain',
             'default-project',
             'ip-fabric',
             '__default__',
             'cn-jenkins-deploy-platform-ansible-os-2962-3'],
 'oper': 'UPDATE',
 'request-id': 'req-5fb1a737-34e3-4bdb-9610-08a6fc253c03',
 'type': 'bgp_router',
 'uuid': 'c9707b2d-c267-4c7b-8c0c-51d3e77bb5e1'}
03/27/2025 00:49:10.970 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain',
             'default-project',
             'ip-fabric',
             '__default__',
             'cn-jenkins-deploy-platform-ansible-os-2962-2'],
 'oper': 'UPDATE-IMPLICIT',
 'request-id': 'req-5fb1a737-34e3-4bdb-9610-08a6fc253c03',
 'type': 'bgp_router',
 'uuid': 'ea923f19-c67a-41d6-ac6f-07a876f3cfb3'}
03/27/2025 00:49:11.034 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain',
             'default-project',
             'ip-fabric',
             '__default__',
             'cn-jenkins-deploy-platform-ansible-os-2962-3'],
 'oper': 'UPDATE',
 'request-id': 'req-abf869a6-5398-4af3-9a0c-62b7b86d478c',
 'type': 'bgp_router',
 'uuid': 'c9707b2d-c267-4c7b-8c0c-51d3e77bb5e1'}
03/27/2025 00:49:11.041 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain',
             'default-project',
             'ip-fabric',
             '__default__',
             'cn-jenkins-deploy-platform-ansible-os-2962-1'],
 'oper': 'UPDATE-IMPLICIT',
 'request-id': 'req-abf869a6-5398-4af3-9a0c-62b7b86d478c',
 'type': 'bgp_router',
 'uuid': '69bdfa77-97e8-4d8e-b806-a021e196c64b'}
03/27/2025 00:49:12.023 7f5a9e625978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:49:12.024 7f5a9e625978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:49:12.024 7f5a9e625978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:49:12.024 7f5a9e625978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:12.025 7f5a9e625978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:49:12.025 7f5a9e625978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:12.025 7f5a9e625978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:49:12.025 7f5a9e625978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:49:12.025 7f5a9e625978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:12.025 7f5a9e625978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:49:12.025 7f5a9e625978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:12.025 7f5a9e625978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:12.026 7f5a9e625978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.43:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:49:12.026 7f5a9e625978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:16.025 7f5a9e625978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:49:16.027 7f5a9e625978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:49:16.027 7f5a9e625978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:49:16.027 7f5a9e625978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:16.027 7f5a9e625978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:49:16.027 7f5a9e625978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:16.027 7f5a9e625978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:49:16.028 7f5a9e625978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:49:16.028 7f5a9e625978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:16.028 7f5a9e625978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:49:16.028 7f5a9e625978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:16.028 7f5a9e625978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:16.028 7f5a9e625978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.27:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:49:16.028 7f5a9e625978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:20.028 7f5a9e625978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:49:20.029 7f5a9e625978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:49:20.029 7f5a9e625978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:49:20.029 7f5a9e625978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:20.029 7f5a9e625978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:49:20.030 7f5a9e625978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:20.030 7f5a9e625978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:49:20.030 7f5a9e625978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:49:20.030 7f5a9e625978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:20.030 7f5a9e625978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:49:20.030 7f5a9e625978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:20.030 7f5a9e625978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:20.030 7f5a9e625978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.36:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:49:20.030 7f5a9e625978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:24.035 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:49:24.036 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:49:24.036 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:49:24.036 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:24.036 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:49:24.036 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:24.036 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:49:24.036 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:49:24.036 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:24.036 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:49:24.036 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:24.037 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:24.037 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.43:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:49:24.037 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:28.037 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:49:28.039 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:49:28.039 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:49:28.039 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:28.039 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:49:28.039 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:28.039 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:49:28.040 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:49:28.040 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:28.040 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:49:28.040 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:28.040 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:28.040 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.27:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:49:28.040 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:32.040 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:49:32.042 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:49:32.043 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:49:32.043 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:32.043 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:49:32.043 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:32.043 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:49:32.044 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:49:32.044 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:32.044 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:49:32.044 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:32.044 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:32.044 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.36:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:49:32.044 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:36.043 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:49:36.044 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:49:36.045 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:49:36.045 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:36.045 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:49:36.045 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:36.045 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:49:36.045 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:49:36.045 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:36.045 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:49:36.045 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:36.046 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:36.046 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.43:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:49:36.046 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:40.047 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:49:40.048 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:49:40.048 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:49:40.048 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:40.048 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:49:40.049 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:40.049 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:49:40.049 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:49:40.049 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:40.049 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:49:40.049 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:40.049 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:40.049 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.27:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:49:40.049 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:44.050 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:49:44.052 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:49:44.052 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:49:44.052 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:44.052 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:49:44.052 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:44.052 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:49:44.053 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:49:44.053 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:44.053 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:49:44.053 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:44.053 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:44.053 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.36:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:49:44.053 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:48.053 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:49:48.056 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:49:48.056 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:49:48.056 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:48.057 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:49:48.057 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:48.057 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:49:48.057 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:49:48.057 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:48.057 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:49:48.057 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:48.057 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:48.057 7f5a9d43f8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.43:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:49:48.057 7f5a9d43f8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:52.057 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:49:52.060 7f5a9d4e3ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:49:52.060 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:49:52.060 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:52.061 7f5a9d4e3ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:49:52.061 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:52.061 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:49:52.061 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:49:52.061 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:52.061 7f5a9d4e3ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:49:52.061 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:52.061 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:52.061 7f5a9d4e3ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.27:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:49:52.062 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:56.061 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:49:56.063 7f5a9d4e3ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:49:56.063 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:49:56.063 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:56.063 7f5a9d4e3ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:49:56.063 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:49:56.063 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:49:56.064 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:49:56.064 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:56.064 7f5a9d4e3ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:49:56.064 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:56.064 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:49:56.064 7f5a9d4e3ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.36:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:49:56.064 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:00.064 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:50:00.065 7f5a9d4e3ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:50:00.066 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:50:00.066 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:00.066 7f5a9d4e3ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:50:00.066 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:00.066 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:50:00.067 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:50:00.067 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:00.067 7f5a9d4e3ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:50:00.067 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:00.067 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:00.067 7f5a9d4e3ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.43:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:50:00.067 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:04.066 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:50:04.068 7f5a9d4e3ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:50:04.068 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:50:04.068 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:04.068 7f5a9d4e3ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:50:04.068 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:04.069 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:50:04.069 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:50:04.069 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:04.069 7f5a9d4e3ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:50:04.069 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:04.069 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:04.069 7f5a9d4e3ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.27:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:50:04.069 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:08.068 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:50:08.070 7f5a9d4e3ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:50:08.070 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:50:08.070 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:08.070 7f5a9d4e3ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:50:08.071 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:08.071 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:50:08.071 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:50:08.071 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:08.071 7f5a9d4e3ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:50:08.071 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:08.071 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:08.071 7f5a9d4e3ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.36:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:50:08.071 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:12.071 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:50:12.072 7f5a9d4e3ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:50:12.072 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:50:12.072 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:12.072 7f5a9d4e3ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:50:12.072 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:12.072 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:50:12.073 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:50:12.073 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:12.073 7f5a9d4e3ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:50:12.073 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:12.073 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:12.073 7f5a9d4e3ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.43:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:50:12.073 7f5a9d4e3ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:12.210 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config',
             'cn-jenkins-deploy-platform-ansible-os-2962-3.'],
 'obj_dict': {'database_node_ip_address': '10.0.0.43',
              'display_name': 'cn-jenkins-deploy-platform-ansible-os-2962-3.',
              'fq_name': ['default-global-system-config',
                          'cn-jenkins-deploy-platform-ansible-os-2962-3.'],
              'id_perms': {'created': '2025-03-27T00:50:12.186332',
                           'creator': None,
                           'description': None,
                           'enable': True,
                           'last_modified': '2025-03-27T00:50:12.186332',
                           'permissions': {'group': 'member',
                                           'group_access': 7,
                                           'other_access': 7,
                                           'owner': 'admin',
                                           'owner_access': 7},
                           'user_visible': True,
                           'uuid': {'uuid_lslong': 9487737700591217163,
                                    'uuid_mslong': 1635632600160027627}},
              'parent_type': 'global-system-config',
              'parent_uuid': 'd46c1ff2-1fd9-4f91-afa4-a8c9ef3fdaac',
              'perms2': {'global_access': 0,
                         'owner': 'cloud-admin',
                         'owner_access': 7,
                         'share': []},
              'uuid': '16b2ef2f-1396-43eb-83ab-372ba5d22a0b'},
 'oper': 'CREATE',
 'request-id': 'req-84cd6a9e-c527-45fe-8a57-4768ff32096b',
 'type': 'database_node',
 'uuid': '16b2ef2f-1396-43eb-83ab-372ba5d22a0b'}
03/27/2025 00:50:12.233 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config',
             'cn-jenkins-deploy-platform-ansible-os-2962-2.'],
 'obj_dict': {'database_node_ip_address': '10.0.0.27',
              'display_name': 'cn-jenkins-deploy-platform-ansible-os-2962-2.',
              'fq_name': ['default-global-system-config',
                          'cn-jenkins-deploy-platform-ansible-os-2962-2.'],
              'id_perms': {'created': '2025-03-27T00:50:12.207881',
                           'creator': None,
                           'description': None,
                           'enable': True,
                           'last_modified': '2025-03-27T00:50:12.207881',
                           'permissions': {'group': 'member',
                                           'group_access': 7,
                                           'other_access': 7,
                                           'owner': 'admin',
                                           'owner_access': 7},
                           'user_visible': True,
                           'uuid': {'uuid_lslong': 9363181450414018474,
                                    'uuid_mslong': 107593575495453756}},
              'parent_type': 'global-system-config',
              'parent_uuid': 'd46c1ff2-1fd9-4f91-afa4-a8c9ef3fdaac',
              'perms2': {'global_access': 0,
                         'owner': 'cloud-admin',
                         'owner_access': 7,
                         'share': []},
              'uuid': '017e3fc9-6f85-4c3c-81f0-b3ebbf744faa'},
 'oper': 'CREATE',
 'request-id': 'req-e4c06100-c682-41f2-bf3f-4dd863e94cd0',
 'type': 'database_node',
 'uuid': '017e3fc9-6f85-4c3c-81f0-b3ebbf744faa'}
03/27/2025 00:50:12.274 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config',
             'cn-jenkins-deploy-platform-ansible-os-2962-1.'],
 'obj_dict': {'database_node_ip_address': '10.0.0.36',
              'display_name': 'cn-jenkins-deploy-platform-ansible-os-2962-1.',
              'fq_name': ['default-global-system-config',
                          'cn-jenkins-deploy-platform-ansible-os-2962-1.'],
              'id_perms': {'created': '2025-03-27T00:50:12.251019',
                           'creator': None,
                           'description': None,
                           'enable': True,
                           'last_modified': '2025-03-27T00:50:12.251019',
                           'permissions': {'group': 'member',
                                           'group_access': 7,
                                           'other_access': 7,
                                           'owner': 'admin',
                                           'owner_access': 7},
                           'user_visible': True,
                           'uuid': {'uuid_lslong': 11531823049461558834,
                                    'uuid_mslong': 18213758889343601421}},
              'parent_type': 'global-system-config',
              'parent_uuid': 'd46c1ff2-1fd9-4f91-afa4-a8c9ef3fdaac',
              'perms2': {'global_access': 0,
                         'owner': 'cloud-admin',
                         'owner_access': 7,
                         'share': []},
              'uuid': 'fcc44535-8ff0-4b0d-a009-43f71bd16e32'},
 'oper': 'CREATE',
 'request-id': 'req-d67fbffa-a78b-434a-b98f-2c69e586cd2d',
 'type': 'database_node',
 'uuid': 'fcc44535-8ff0-4b0d-a009-43f71bd16e32'}
03/27/2025 00:50:16.073 7f5a9d4dd438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:50:16.074 7f5a9d4dd438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:50:16.074 7f5a9d4dd438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:50:16.075 7f5a9d4dd438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:16.075 7f5a9d4dd438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:50:16.075 7f5a9d4dd438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:16.075 7f5a9d4dd438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:50:16.075 7f5a9d4dd438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:50:16.075 7f5a9d4dd438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:16.075 7f5a9d4dd438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:50:16.075 7f5a9d4dd438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:16.076 7f5a9d4dd438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:16.076 7f5a9d4dd438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.27:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:50:16.076 7f5a9d4dd438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:20.076 7f5a9d4dd438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:50:20.077 7f5a9d4dd438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:50:20.077 7f5a9d4dd438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:50:20.077 7f5a9d4dd438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:20.078 7f5a9d4dd438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:50:20.078 7f5a9d4dd438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:20.078 7f5a9d4dd438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:50:20.078 7f5a9d4dd438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:50:20.078 7f5a9d4dd438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:20.078 7f5a9d4dd438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:50:20.078 7f5a9d4dd438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:20.078 7f5a9d4dd438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:20.079 7f5a9d4dd438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.36:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:50:20.079 7f5a9d4dd438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:24.079 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:50:24.080 7f5a9d27d8d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:50:24.080 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:50:24.080 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:24.080 7f5a9d27d8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:50:24.080 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:24.081 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:50:24.081 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:50:24.081 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:24.081 7f5a9d27d8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:50:24.081 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:24.081 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:24.081 7f5a9d27d8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.43:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:50:24.081 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:28.082 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:50:28.084 7f5a9d27d8d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:50:28.085 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:50:28.085 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:28.085 7f5a9d27d8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:50:28.085 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:28.085 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:50:28.086 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:50:28.086 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:28.086 7f5a9d27d8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:50:28.086 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:28.086 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:28.086 7f5a9d27d8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.27:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:50:28.086 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:32.086 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:50:32.088 7f5a9d27d8d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:50:32.088 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:50:32.088 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:32.088 7f5a9d27d8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:50:32.089 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:32.089 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:50:32.089 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:50:32.089 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:32.089 7f5a9d27d8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:50:32.089 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:32.089 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:32.089 7f5a9d27d8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.36:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:50:32.089 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:36.089 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:50:36.091 7f5a9d27d8d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:50:36.091 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:50:36.091 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:36.091 7f5a9d27d8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:50:36.091 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:36.091 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:50:36.092 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:50:36.092 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:36.092 7f5a9d27d8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:50:36.092 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:36.092 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:36.092 7f5a9d27d8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.43:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:50:36.092 7f5a9d27d8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:40.092 7f5a9e61df60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:50:40.095 7f5a9e61df60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:50:40.095 7f5a9e61df60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:50:40.095 7f5a9e61df60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:40.095 7f5a9e61df60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:50:40.095 7f5a9e61df60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:40.095 7f5a9e61df60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:50:40.096 7f5a9e61df60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:50:40.096 7f5a9e61df60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:40.096 7f5a9e61df60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:50:40.096 7f5a9e61df60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:40.096 7f5a9e61df60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:40.096 7f5a9e61df60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.27:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:50:40.096 7f5a9e61df60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:44.096 7f5a9e61df60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:50:44.098 7f5a9e61df60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:50:44.098 7f5a9e61df60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:50:44.098 7f5a9e61df60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:44.098 7f5a9e61df60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:50:44.098 7f5a9e61df60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:44.098 7f5a9e61df60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:50:44.098 7f5a9e61df60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:50:44.099 7f5a9e61df60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:44.099 7f5a9e61df60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:50:44.099 7f5a9e61df60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:44.099 7f5a9e61df60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:44.099 7f5a9e61df60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.36:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:50:44.099 7f5a9e61df60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:48.099 7f5a9e61df60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:50:48.101 7f5a9e61df60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:50:48.101 7f5a9e61df60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:50:48.101 7f5a9e61df60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:48.101 7f5a9e61df60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:50:48.101 7f5a9e61df60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:48.101 7f5a9e61df60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:50:48.102 7f5a9e61df60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:50:48.102 7f5a9e61df60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:48.102 7f5a9e61df60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:50:48.102 7f5a9e61df60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:48.102 7f5a9e61df60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:48.102 7f5a9e61df60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.43:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:50:48.102 7f5a9e61df60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:52.101 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:50:52.102 7f5a9d4d52e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:50:52.102 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:50:52.102 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:52.103 7f5a9d4d52e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:50:52.103 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:52.103 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:50:52.103 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:50:52.103 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:52.103 7f5a9d4d52e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:50:52.103 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:52.103 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:52.103 7f5a9d4d52e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.27:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:50:52.104 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:56.103 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:50:56.104 7f5a9d4d52e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:50:56.105 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:50:56.105 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:56.105 7f5a9d4d52e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:50:56.105 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:50:56.105 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:50:56.105 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:50:56.105 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:56.105 7f5a9d4d52e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:50:56.106 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:56.106 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:50:56.106 7f5a9d4d52e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.36:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:50:56.106 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:51:00.106 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:51:00.107 7f5a9d4d52e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:51:00.108 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:51:00.108 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:51:00.108 7f5a9d4d52e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:51:00.108 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:51:00.108 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:51:00.108 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:51:00.108 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:51:00.108 7f5a9d4d52e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.43:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:51:00.108 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:51:00.109 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:51:00.109 7f5a9d4d52e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.43:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:51:00.109 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:51:04.109 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:51:04.111 7f5a9d4d52e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
03/27/2025 00:51:04.111 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:51:04.112 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:51:04.112 7f5a9d4d52e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:51:04.112 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:51:04.112 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
03/27/2025 00:51:04.113 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
03/27/2025 00:51:04.113 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:51:04.114 7f5a9d4d52e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.27:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
03/27/2025 00:51:04.114 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:51:04.114 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:51:04.114 7f5a9d4d52e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0  client_info = <<  status = Idle  successful_connections = 0  pid = 140  http_port = 8096  start_time = 1743036427944866  collector_name =   collector_ip = 10.0.0.27:8086  collector_list = [ 10.0.0.43:8086, 10.0.0.27:8086, 10.0.0.36:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
03/27/2025 00:51:04.114 7f5a9d4d52e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
03/27/2025 00:51:07.562 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config',
             'cn-jenkins-deploy-platform-ansible-os-2962-3.'],
 'obj_dict': {'analytics_node_ip_address': '10.0.0.43',
              'display_name': 'cn-jenkins-deploy-platform-ansible-os-2962-3.',
              'fq_name': ['default-global-system-config',
                          'cn-jenkins-deploy-platform-ansible-os-2962-3.'],
              'id_perms': {'created': '2025-03-27T00:51:07.541798',
                           'creator': None,
                           'description': None,
                           'enable': True,
                           'last_modified': '2025-03-27T00:51:07.541798',
                           'permissions': {'group': 'member',
                                           'group_access': 7,
                                           'other_access': 7,
                                           'owner': 'admin',
                                           'owner_access': 7},
                           'user_visible': True,
                           'uuid': {'uuid_lslong': 11177307489897744986,
                                    'uuid_mslong': 7883022446536641544}},
              'parent_type': 'global-system-config',
              'parent_uuid': 'd46c1ff2-1fd9-4f91-afa4-a8c9ef3fdaac',
              'perms2': {'global_access': 0,
                         'owner': 'cloud-admin',
                         'owner_access': 7,
                         'share': []},
              'uuid': '6d661f26-2a3f-4c08-9b1d-c5f130ab0a5a'},
 'oper': 'CREATE',
 'request-id': 'req-7d17e5fc-9df3-4ffd-8986-8952e1d9e8f7',
 'type': 'analytics_node',
 'uuid': '6d661f26-2a3f-4c08-9b1d-c5f130ab0a5a'}
03/27/2025 00:51:07.677 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config',
             'cn-jenkins-deploy-platform-ansible-os-2962-2.'],
 'obj_dict': {'analytics_node_ip_address': '10.0.0.27',
              'display_name': 'cn-jenkins-deploy-platform-ansible-os-2962-2.',
              'fq_name': ['default-global-system-config',
                          'cn-jenkins-deploy-platform-ansible-os-2962-2.'],
              'id_perms': {'created': '2025-03-27T00:51:07.666038',
                           'creator': None,
                           'description': None,
                           'enable': True,
                           'last_modified': '2025-03-27T00:51:07.666038',
                           'permissions': {'group': 'member',
                                           'group_access': 7,
                                           'other_access': 7,
                                           'owner': 'admin',
                                           'owner_access': 7},
                           'user_visible': True,
                           'uuid': {'uuid_lslong': 9275901355391550603,
                                    'uuid_mslong': 7351039259150601471}},
              'parent_type': 'global-system-config',
              'parent_uuid': 'd46c1ff2-1fd9-4f91-afa4-a8c9ef3fdaac',
              'perms2': {'global_access': 0,
                         'owner': 'cloud-admin',
                         'owner_access': 7,
                         'share': []},
              'uuid': '66042342-3b9f-4cff-80ba-9f230bad848b'},
 'oper': 'CREATE',
 'request-id': 'req-e7cca41a-55c3-42b1-b9fa-8dda7fd97259',
 'type': 'analytics_node',
 'uuid': '66042342-3b9f-4cff-80ba-9f230bad848b'}
03/27/2025 00:51:07.748 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config',
             'cn-jenkins-deploy-platform-ansible-os-2962-1.'],
 'obj_dict': {'analytics_node_ip_address': '10.0.0.36',
              'display_name': 'cn-jenkins-deploy-platform-ansible-os-2962-1.',
              'fq_name': ['default-global-system-config',
                          'cn-jenkins-deploy-platform-ansible-os-2962-1.'],
              'id_perms': {'created': '2025-03-27T00:51:07.733909',
                           'creator': None,
                           'description': None,
                           'enable': True,
                           'last_modified': '2025-03-27T00:51:07.733909',
                           'permissions': {'group': 'member',
                                           'group_access': 7,
                                           'other_access': 7,
                                           'owner': 'admin',
                                           'owner_access': 7},
                           'user_visible': True,
                           'uuid': {'uuid_lslong': 11430691180620364802,
                                    'uuid_mslong': 7213681714200661724}},
              'parent_type': 'global-system-config',
              'parent_uuid': 'd46c1ff2-1fd9-4f91-afa4-a8c9ef3fdaac',
              'perms2': {'global_access': 0,
                         'owner': 'cloud-admin',
                         'owner_access': 7,
                         'share': []},
              'uuid': '641c254c-a1a9-4edc-9ea1-f910fe113c02'},
 'oper': 'CREATE',
 'request-id': 'req-18990187-d4e5-43d1-8f93-39c56548f0b0',
 'type': 'analytics_node',
 'uuid': '641c254c-a1a9-4edc-9ea1-f910fe113c02'}
03/27/2025 00:51:08.111 7f5aa4c9bc18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
03/27/2025 00:51:08.113 7f5aa4c9bc18 [contrail-device-manager] [INFO]: Session Event: TCP Connected
03/27/2025 00:51:08.113 7f5aa4c9bc18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
03/27/2025 00:51:08.116 7f5aa4c9bc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:51:08.116 7f5aa4c9bc18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2962-3.  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.36:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.36:5673, 10.0.0.27:5673, 10.0.0.43:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.36:9041, 10.0.0.27:9041, 10.0.0.43:9041,  ]  status = Up  description =   >>, <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.36:2181, 10.0.0.27:2181, 10.0.0.43:2181,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 10.0.0.36,10.0.0.27,10.0.0.43:8082,  ]  status = Up  description = ApiServer Connection State updated  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
03/27/2025 00:51:08.116 7f5aa4c9bc18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
03/27/2025 00:51:08.116 7f5aa4c9bc18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect]
03/27/2025 00:51:08.116 7f5aa4c9bc18 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19
03/27/2025 00:51:08.116 7f5aa4c9bc18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit]
03/27/2025 00:51:08.117 7f5aa4c9bc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
03/27/2025 00:51:08.117 7f5aa4c9bc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
03/27/2025 00:51:08.117 7f5a9d2aab70 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG]
03/27/2025 00:51:08.118 7f5a9d2aab70 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID]
03/27/2025 00:51:20.331 7f5a9d1d4390 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001016
03/27/2025 00:51:20.332 7f5a9d1d4390 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient]
03/27/2025 00:51:20.333 7f5a9f4db588 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit]
03/27/2025 00:51:20.333 7f5a9f4db588 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 0
03/27/2025 00:51:20.333 7f5a9f4db588 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 146> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 00:51:20.333 7f5a9f4db588 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0: 61> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
03/27/2025 00:51:20.333 7f5a9f4db588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established]
03/27/2025 00:51:20.333 7f5a9d49f780 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG]
03/27/2025 00:51:20.335 7f5a9d49f780 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID]
03/27/2025 00:51:29.513 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config',
             'cn-jenkins-deploy-platform-ansible-os-2962-3.'],
 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.43',
              'display_name': 'cn-jenkins-deploy-platform-ansible-os-2962-3.',
              'fq_name': ['default-global-system-config',
                          'cn-jenkins-deploy-platform-ansible-os-2962-3.'],
              'id_perms': {'created': '2025-03-27T00:51:29.475040',
                           'creator': None,
                           'description': None,
                           'enable': True,
                           'last_modified': '2025-03-27T00:51:29.475040',
                           'permissions': {'group': 'member',
                                           'group_access': 7,
                                           'other_access': 7,
                                           'owner': 'admin',
                                           'owner_access': 7},
                           'user_visible': True,
                           'uuid': {'uuid_lslong': 9521436490660335202,
                                    'uuid_mslong': 5017360007071155132}},
              'parent_type': 'global-system-config',
              'parent_uuid': 'd46c1ff2-1fd9-4f91-afa4-a8c9ef3fdaac',
              'perms2': {'global_access': 0,
                         'owner': 'cloud-admin',
                         'owner_access': 7,
                         'share': []},
              'uuid': '45a13e57-e3ba-43bc-8422-f00aa1404e62'},
 'oper': 'CREATE',
 'request-id': 'req-92d067e6-4c77-4294-9bf4-c09fb8e67c49',
 'type': 'analytics_alarm_node',
 'uuid': '45a13e57-e3ba-43bc-8422-f00aa1404e62'}
03/27/2025 00:51:29.635 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config',
             'cn-jenkins-deploy-platform-ansible-os-2962-2.'],
 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.27',
              'display_name': 'cn-jenkins-deploy-platform-ansible-os-2962-2.',
              'fq_name': ['default-global-system-config',
                          'cn-jenkins-deploy-platform-ansible-os-2962-2.'],
              'id_perms': {'created': '2025-03-27T00:51:29.617130',
                           'creator': None,
                           'description': None,
                           'enable': True,
                           'last_modified': '2025-03-27T00:51:29.617130',
                           'permissions': {'group': 'member',
                                           'group_access': 7,
                                           'other_access': 7,
                                           'owner': 'admin',
                                           'owner_access': 7},
                           'user_visible': True,
                           'uuid': {'uuid_lslong': 9630612095501955226,
                                    'uuid_mslong': 10230060178534516879}},
              'parent_type': 'global-system-config',
              'parent_uuid': 'd46c1ff2-1fd9-4f91-afa4-a8c9ef3fdaac',
              'perms2': {'global_access': 0,
                         'owner': 'cloud-admin',
                         'owner_access': 7,
                         'share': []},
              'uuid': '8df87987-fc6b-488f-85a6-ceacf9f9709a'},
 'oper': 'CREATE',
 'request-id': 'req-236c469e-995b-41fc-89d5-11c5ff6a4bbd',
 'type': 'analytics_alarm_node',
 'uuid': '8df87987-fc6b-488f-85a6-ceacf9f9709a'}
03/27/2025 00:51:29.818 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config',
             'cn-jenkins-deploy-platform-ansible-os-2962-1.'],
 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.36',
              'display_name': 'cn-jenkins-deploy-platform-ansible-os-2962-1.',
              'fq_name': ['default-global-system-config',
                          'cn-jenkins-deploy-platform-ansible-os-2962-1.'],
              'id_perms': {'created': '2025-03-27T00:51:29.798134',
                           'creator': None,
                           'description': None,
                           'enable': True,
                           'last_modified': '2025-03-27T00:51:29.798134',
                           'permissions': {'group': 'member',
                                           'group_access': 7,
                                           'other_access': 7,
                                           'owner': 'admin',
                                           'owner_access': 7},
                           'user_visible': True,
                           'uuid': {'uuid_lslong': 12674856626740439420,
                                    'uuid_mslong': 873381369293521237}},
              'parent_type': 'global-system-config',
              'parent_uuid': 'd46c1ff2-1fd9-4f91-afa4-a8c9ef3fdaac',
              'perms2': {'global_access': 0,
                         'owner': 'cloud-admin',
                         'owner_access': 7,
                         'share': []},
              'uuid': '0c1edfba-5e2f-4155-afe6-22f29fdea97c'},
 'oper': 'CREATE',
 'request-id': 'req-abec5105-ddce-4b15-9e2b-484cfd9d4bff',
 'type': 'analytics_alarm_node',
 'uuid': '0c1edfba-5e2f-4155-afe6-22f29fdea97c'}
03/27/2025 00:51:31.471 7f5a9d1d4390 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001297
03/27/2025 00:51:31.471 7f5a9d1d4390 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient]
03/27/2025 00:51:31.472 7f5a9e631cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established]
03/27/2025 00:51:31.472 7f5a9e631cc0 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 2
03/27/2025 00:51:31.472 7f5a9e631cc0 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 00:51:31.472 7f5a9e631cc0 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.:Config:contrail-device-manager:0: 62> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
03/27/2025 00:51:48.985 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config',
             'cn-jenkins-deploy-platform-ansible-os-2962-3.'],
 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.43',
              'display_name': 'cn-jenkins-deploy-platform-ansible-os-2962-3.',
              'fq_name': ['default-global-system-config',
                          'cn-jenkins-deploy-platform-ansible-os-2962-3.'],
              'id_perms': {'created': '2025-03-27T00:51:48.978142',
                           'creator': None,
                           'description': None,
                           'enable': True,
                           'last_modified': '2025-03-27T00:51:48.978142',
                           'permissions': {'group': 'member',
                                           'group_access': 7,
                                           'other_access': 7,
                                           'owner': 'admin',
                                           'owner_access': 7},
                           'user_visible': True,
                           'uuid': {'uuid_lslong': 12069753490711022638,
                                    'uuid_mslong': 14141169615412348150}},
              'parent_type': 'global-system-config',
              'parent_uuid': 'd46c1ff2-1fd9-4f91-afa4-a8c9ef3fdaac',
              'perms2': {'global_access': 0,
                         'owner': 'cloud-admin',
                         'owner_access': 7,
                         'share': []},
              'uuid': 'c43f86d7-9333-44f6-a780-60d9fc5c8c2e'},
 'oper': 'CREATE',
 'request-id': 'req-f0c6c917-9f00-4faa-8dff-eccf9b228b65',
 'type': 'analytics_snmp_node',
 'uuid': 'c43f86d7-9333-44f6-a780-60d9fc5c8c2e'}
03/27/2025 00:51:50.031 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config',
             'cn-jenkins-deploy-platform-ansible-os-2962-2.'],
 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.27',
              'display_name': 'cn-jenkins-deploy-platform-ansible-os-2962-2.',
              'fq_name': ['default-global-system-config',
                          'cn-jenkins-deploy-platform-ansible-os-2962-2.'],
              'id_perms': {'created': '2025-03-27T00:51:50.005658',
                           'creator': None,
                           'description': None,
                           'enable': True,
                           'last_modified': '2025-03-27T00:51:50.005658',
                           'permissions': {'group': 'member',
                                           'group_access': 7,
                                           'other_access': 7,
                                           'owner': 'admin',
                                           'owner_access': 7},
                           'user_visible': True,
                           'uuid': {'uuid_lslong': 11338931040338764882,
                                    'uuid_mslong': 2855078791180536155}},
              'parent_type': 'global-system-config',
              'parent_uuid': 'd46c1ff2-1fd9-4f91-afa4-a8c9ef3fdaac',
              'perms2': {'global_access': 0,
                         'owner': 'cloud-admin',
                         'owner_access': 7,
                         'share': []},
              'uuid': '279f4708-a211-495b-9d5b-f9b4780ae452'},
 'oper': 'CREATE',
 'request-id': 'req-dd36b5f7-4129-4e01-ac1b-3c4168cfb6a1',
 'type': 'analytics_snmp_node',
 'uuid': '279f4708-a211-495b-9d5b-f9b4780ae452'}
03/27/2025 00:51:50.373 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config',
             'cn-jenkins-deploy-platform-ansible-os-2962-1.'],
 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.36',
              'display_name': 'cn-jenkins-deploy-platform-ansible-os-2962-1.',
              'fq_name': ['default-global-system-config',
                          'cn-jenkins-deploy-platform-ansible-os-2962-1.'],
              'id_perms': {'created': '2025-03-27T00:51:50.360198',
                           'creator': None,
                           'description': None,
                           'enable': True,
                           'last_modified': '2025-03-27T00:51:50.360198',
                           'permissions': {'group': 'member',
                                           'group_access': 7,
                                           'other_access': 7,
                                           'owner': 'admin',
                                           'owner_access': 7},
                           'user_visible': True,
                           'uuid': {'uuid_lslong': 10633859207708111103,
                                    'uuid_mslong': 251612456912833810}},
              'parent_type': 'global-system-config',
              'parent_uuid': 'd46c1ff2-1fd9-4f91-afa4-a8c9ef3fdaac',
              'perms2': {'global_access': 0,
                         'owner': 'cloud-admin',
                         'owner_access': 7,
                         'share': []},
              'uuid': '037de832-4b5a-4d12-9393-0e9bdaf570ff'},
 'oper': 'CREATE',
 'request-id': 'req-91e02674-2081-4b3d-8333-fdfb3ac83e77',
 'type': 'analytics_snmp_node',
 'uuid': '037de832-4b5a-4d12-9393-0e9bdaf570ff'}
03/27/2025 00:53:46.874 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'],
 'oper': 'UPDATE',
 'request-id': 'req-e57e5409-58c1-412f-93c5-18c7323f053c',
 'type': 'global_system_config',
 'uuid': 'd46c1ff2-1fd9-4f91-afa4-a8c9ef3fdaac'}
03/27/2025 00:53:46.950 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'],
 'oper': 'UPDATE',
 'request-id': 'req-a3b183dd-e79e-4ea4-85a5-dbb88df13f5f',
 'type': 'global_system_config',
 'uuid': 'd46c1ff2-1fd9-4f91-afa4-a8c9ef3fdaac'}
03/27/2025 00:53:47.015 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config',
             'an-jenkins-deploy-platform-ansible-os-2962-1'],
 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-2962-1',
              'fq_name': ['default-global-system-config',
                          'an-jenkins-deploy-platform-ansible-os-2962-1'],
              'id_perms': {'created': '2025-03-27T00:53:47.005997',
                           'creator': None,
                           'description': None,
                           'enable': True,
                           'last_modified': '2025-03-27T00:53:47.005997',
                           'permissions': {'group': 'member',
                                           'group_access': 7,
                                           'other_access': 7,
                                           'owner': 'admin',
                                           'owner_access': 7},
                           'user_visible': True,
                           'uuid': {'uuid_lslong': 11340839921940959295,
                                    'uuid_mslong': 9596602363801586140}},
              'parent_type': 'global-system-config',
              'parent_uuid': 'd46c1ff2-1fd9-4f91-afa4-a8c9ef3fdaac',
              'perms2': {'global_access': 0,
                         'owner': 'cloud-admin',
                         'owner_access': 7,
                         'share': []},
              'uuid': '852dfb01-4111-49dc-9d62-c1d299dc103f',
              'virtual_router_dpdk_enabled': False,
              'virtual_router_ip_address': '10.20.0.5'},
 'oper': 'CREATE',
 'request-id': 'req-8242eb6d-a9af-4b2f-9f14-a370dd2984db',
 'type': 'virtual_router',
 'uuid': '852dfb01-4111-49dc-9d62-c1d299dc103f'}
03/27/2025 00:53:47.079 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config',
             'an-jenkins-deploy-platform-ansible-os-2962-2'],
 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-2962-2',
              'fq_name': ['default-global-system-config',
                          'an-jenkins-deploy-platform-ansible-os-2962-2'],
              'id_perms': {'created': '2025-03-27T00:53:47.070624',
                           'creator': None,
                           'description': None,
                           'enable': True,
                           'last_modified': '2025-03-27T00:53:47.070624',
                           'permissions': {'group': 'member',
                                           'group_access': 7,
                                           'other_access': 7,
                                           'owner': 'admin',
                                           'owner_access': 7},
                           'user_visible': True,
                           'uuid': {'uuid_lslong': 11508548855672746410,
                                    'uuid_mslong': 7656434137105450911}},
              'parent_type': 'global-system-config',
              'parent_uuid': 'd46c1ff2-1fd9-4f91-afa4-a8c9ef3fdaac',
              'perms2': {'global_access': 0,
                         'owner': 'cloud-admin',
                         'owner_access': 7,
                         'share': []},
              'uuid': '6a411e48-3c58-479f-9fb6-9435969a31aa',
              'virtual_router_dpdk_enabled': False,
              'virtual_router_ip_address': '10.20.0.11'},
 'oper': 'CREATE',
 'request-id': 'req-9a75b3de-34de-4076-9ffd-09e387ec274a',
 'type': 'virtual_router',
 'uuid': '6a411e48-3c58-479f-9fb6-9435969a31aa'}
03/27/2025 00:53:47.395 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config',
             'an-jenkins-deploy-platform-ansible-os-2962-1',
             'vhost0'],
 'obj_dict': {'display_name': 'vhost0',
              'fq_name': ['default-global-system-config',
                          'an-jenkins-deploy-platform-ansible-os-2962-1',
                          'vhost0'],
              'id_perms': {'created': '2025-03-27T00:53:47.377230',
                           'creator': None,
                           'description': None,
                           'enable': True,
                           'last_modified': '2025-03-27T00:53:47.377230',
                           'permissions': {'group': 'member',
                                           'group_access': 7,
                                           'other_access': 7,
                                           'owner': 'admin',
                                           'owner_access': 7},
                           'user_visible': True,
                           'uuid': {'uuid_lslong': 13505770319972453594,
                                    'uuid_mslong': 12226853092335504853}},
              'parent_type': 'virtual-router',
              'parent_uuid': '852dfb01-4111-49dc-9d62-c1d299dc103f',
              'perms2': {'global_access': 0,
                         'owner': 'cloud-admin',
                         'owner_access': 7,
                         'share': []},
              'uuid': 'a9ae821b-598b-4dd5-bb6e-228ca3f8ccda',
              'virtual_machine_interface_disable_policy': True,
              'virtual_machine_interface_mac_addresses': {'mac_address': ['02:a9:ae:82:1b:59']},
              'virtual_network_refs': [{'to': ['default-domain',
                                               'default-project',
                                               'ip-fabric'],
                                        'uuid': '56430df0-f5c1-4289-9e63-c7e8a940b9c7'}]},
 'oper': 'CREATE',
 'request-id': 'req-132456bb-7532-47e1-9cda-070424a3aa98',
 'type': 'virtual_machine_interface',
 'uuid': 'a9ae821b-598b-4dd5-bb6e-228ca3f8ccda'}
03/27/2025 00:53:47.464 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config',
             'an-jenkins-deploy-platform-ansible-os-2962-1',
             'vhost0'],
 'oper': 'UPDATE',
 'request-id': 'req-132456bb-7532-47e1-9cda-070424a3aa98',
 'type': 'virtual_machine_interface',
 'uuid': 'a9ae821b-598b-4dd5-bb6e-228ca3f8ccda'}
03/27/2025 00:53:47.489 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config',
             'an-jenkins-deploy-platform-ansible-os-2962-2',
             'vhost0'],
 'obj_dict': {'display_name': 'vhost0',
              'fq_name': ['default-global-system-config',
                          'an-jenkins-deploy-platform-ansible-os-2962-2',
                          'vhost0'],
              'id_perms': {'created': '2025-03-27T00:53:47.447274',
                           'creator': None,
                           'description': None,
                           'enable': True,
                           'last_modified': '2025-03-27T00:53:47.447274',
                           'permissions': {'group': 'member',
                                           'group_access': 7,
                                           'other_access': 7,
                                           'owner': 'admin',
                                           'owner_access': 7},
                           'user_visible': True,
                           'uuid': {'uuid_lslong': 12165650254294160315,
                                    'uuid_mslong': 16017501251612462185}},
              'parent_type': 'virtual-router',
              'parent_uuid': '6a411e48-3c58-479f-9fb6-9435969a31aa',
              'perms2': {'global_access': 0,
                         'owner': 'cloud-admin',
                         'owner_access': 7,
                         'share': []},
              'uuid': 'de499886-23bc-4c69-a8d5-12732cd813bb',
              'virtual_machine_interface_disable_policy': True,
              'virtual_machine_interface_mac_addresses': {'mac_address': ['02:de:49:98:86:23']},
              'virtual_network_refs': [{'to': ['default-domain',
                                               'default-project',
                                               'ip-fabric'],
                                        'uuid': '56430df0-f5c1-4289-9e63-c7e8a940b9c7'}]},
 'oper': 'CREATE',
 'request-id': 'req-33382f39-a5cf-40ca-84ac-b26aebe04b9c',
 'type': 'virtual_machine_interface',
 'uuid': 'de499886-23bc-4c69-a8d5-12732cd813bb'}
03/27/2025 00:53:47.536 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config',
             'an-jenkins-deploy-platform-ansible-os-2962-2',
             'vhost0'],
 'oper': 'UPDATE',
 'request-id': 'req-33382f39-a5cf-40ca-84ac-b26aebe04b9c',
 'type': 'virtual_machine_interface',
 'uuid': 'de499886-23bc-4c69-a8d5-12732cd813bb'}
03/27/2025 00:53:47.997 7f5a9d3702e8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'],
 'oper': 'UPDATE',
 'request-id': 'req-e70b29b1-26f4-47de-8c25-e47fc72f2f3a',
 'type': 'access_control_list',
 'uuid': '5b01c67a-9b28-4d46-aec7-b95fe5c90b75'}
03/27/2025 00:54:40.682 7f5aa61e1a20 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 00:55:01.269 7f5aa61e1a20 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 00:55:21.938 7f5aa61e1a20 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 00:55:41.834 7f5a9f4f0630 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 00:56:02.114 7f5aa43a8cc0 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 00:56:23.168 7f5a9d2d7d68 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 00:56:42.964 7f5a9d430e10 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 00:57:02.685 7f5aa518d978 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 00:57:23.740 7f5a9d4a9048 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 00:57:43.487 7f5a9f55b6d8 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 00:58:03.852 7f5a9d491eb8 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 00:58:25.828 7f5a9d280240 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 00:58:45.958 7f5aa4c9bc18 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 00:59:06.016 7f5a9d4212e8 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 00:59:27.364 7f5a9d2d78d0 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 00:59:47.242 7f5a9d2d78d0 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:00:59.162 7f5a9d1d6780 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:01:20.490 7f5a9d2d7240 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:01:41.319 7f5a9d1d4780 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:02:01.652 7f5a9d4a9048 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:02:22.303 7f5a9d29a780 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:02:42.573 7f5a9d1d4630 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:03:03.156 7f5a9e9a4f60 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:03:24.115 7f5a9e9b5048 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:03:44.512 7f5a9d1d4d68 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:04:04.936 7f5aa61e1a20 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:04:25.665 7f5aa4285630 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:04:45.505 7f5aa4285630 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:05:06.484 7f5a9d29a048 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:05:27.445 7f5a9f55b780 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:05:48.130 7f5a9e8aa8d0 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:06:08.901 7f5a9d29ae10 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:06:29.273 7f5a9d2d7240 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:06:49.653 7f5a9d491eb8 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:07:11.170 7f5a9d4e3ac8 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:07:31.467 7f5a9d1d6828 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:07:51.565 7f5a9f55bd68 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:08:12.761 7f5a9d33c390 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:08:33.262 7f5a9d2d7d68 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:08:53.467 7f5a9d29ff60 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:09:14.421 7f5a9e631cc0 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:09:34.449 7f5a9d1d4630 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:09:55.109 7f5a9f4774e0 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:10:15.960 7f5a9f4774e0 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:10:36.520 7f5a9f55ba20 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:10:57.335 7f5a9d33cd68 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:11:18.035 7f5a9d33cd68 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:11:38.421 7f5aa43a8cc0 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:11:59.015 7f5a9d280240 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:12:19.616 7f5a9d280240 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:12:40.469 7f5a9f55bd68 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:13:00.843 7f5a9d29a828 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:13:21.951 7f5aa4285630 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:13:42.263 7f5aa518d978 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:14:01.817 7f5a9d430e10 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:14:22.839 7f5a9f55ba20 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:14:43.175 7f5a9f55ba20 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:14:52.516 7f5a9e61ceb8 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:15:02.004 7f5a9e61ceb8 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map
03/27/2025 01:16:33.686 7f5a9d1d42e8 [contrail-device-manager] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2962-3.: 147> in the [ObjectConfigNode:NodeStatusUVE] map