02/03/2025 00:46:27.284 7f200edb3b08 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 02/03/2025 00:46:27.284 7f200edb3b08 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 02/03/2025 00:46:27.284 7f200edb3b08 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 02/03/2025 00:46:27.284 7f20064c1b70 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 02/03/2025 00:46:27.285 7f20064c1b70 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 00:46:27.285 7f20064c1b70 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/03/2025 00:46:27.285 7f20064c1b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 02/03/2025 00:46:27.285 7f20064c1b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:27.285 7f20064c1b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/03/2025 00:46:27.285 7f20064c1b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:27.285 7f20064c1b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:27.285 7f20064c1b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1738543587285289 collector_name = collector_ip = collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 02/03/2025 00:46:27.285 7f20064c1b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:27.337 7f200edb3b08 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.36:5673//', 'pyamqp://guest:guest@10.0.0.32:5673//', 'pyamqp://guest:guest@10.0.0.30:5673//'] 02/03/2025 00:46:27.346 7f200edb3b08 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 02/03/2025 00:46:27.347 7f200646f198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:27.347 7f200646f198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 02/03/2025 00:46:27.347 7f200646f198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:27.347 7f200646f198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:27.347 7f200646f198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 02/03/2025 00:46:27.347 7f200646f198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:27.351 7f200edb3b08 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 02/03/2025 00:46:27.351 7f200632db70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:27.351 7f200632db70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/03/2025 00:46:27.351 7f200632db70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:27.388 7f200edb3b08 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 02/03/2025 00:46:27.389 7f200646f438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:27.389 7f200646f438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 02/03/2025 00:46:27.389 7f200646f438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:27.393 7f200edb3b08 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 02/03/2025 00:46:27.393 7f200646f198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:27.393 7f200646f198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/03/2025 00:46:27.393 7f200646f198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:27.658 7f200edb3b08 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 02/03/2025 00:46:27.658 7f200edb3b08 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 02/03/2025 00:46:27.658 7f200edb3b08 [contrail-device-manager] [WARNING]: initializing cassandra cluster 02/03/2025 00:46:27.659 7f200edb3b08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 02/03/2025 00:46:27.659 7f200bd32780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 02/03/2025 00:46:27.659 7f200bd32780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 02/03/2025 00:46:27.659 7f2006469e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 02/03/2025 00:46:27.659 7f2006469e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 02/03/2025 00:46:27.660 7f200628d438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:27.660 7f200628d438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 02/03/2025 00:46:27.660 7f200628d438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:27.661 7f200edb3b08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 02/03/2025 00:46:27.668 7f200bd32780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 02/03/2025 00:46:27.668 7f200bd32780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 02/03/2025 00:46:27.668 7f2006469e10 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 02/03/2025 00:46:27.694 7f200edb3b08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 02/03/2025 00:46:27.695 7f2006220630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:27.695 7f2006220630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 02/03/2025 00:46:27.695 7f2006220630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:27.704 7f200edb3b08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 02/03/2025 00:46:27.704 7f2006209390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:27.704 7f2006209390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 02/03/2025 00:46:27.704 7f2006209390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:27.711 7f200edb3b08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 02/03/2025 00:46:27.731 7f200edb3b08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 02/03/2025 00:46:29.228 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 02/03/2025 00:46:30.214 7f200edb3b08 [contrail-device-manager] [WARNING]: table 'obj_uuid_table', created 02/03/2025 00:46:30.220 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 02/03/2025 00:46:31.155 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 02/03/2025 00:46:31.285 7f2006285198 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:46:31.287 7f2006285198 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:46:31.287 7f2006285198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:46:31.287 7f2006285198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:31.287 7f2006285198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 02/03/2025 00:46:31.287 7f2006285198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:31.287 7f2006285198 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:46:31.287 7f2006285198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:46:31.287 7f2006285198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:31.288 7f2006285198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 02/03/2025 00:46:31.288 7f2006285198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:31.288 7f2006285198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:31.288 7f2006285198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:46:31.288 7f2006285198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:32.098 7f200edb3b08 [contrail-device-manager] [WARNING]: table 'obj_uuid_table' fixed with properties {'gc_grace_seconds': 864000} 02/03/2025 00:46:32.104 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 02/03/2025 00:46:34.186 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 02/03/2025 00:46:35.128 7f200edb3b08 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table', created 02/03/2025 00:46:35.134 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 02/03/2025 00:46:35.287 7f2006199c18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:46:35.289 7f2006199c18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:46:35.289 7f2006199c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:46:35.289 7f2006199c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:35.289 7f2006199c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 02/03/2025 00:46:35.289 7f2006199c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:35.289 7f2006199c18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:46:35.289 7f2006199c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:46:35.289 7f2006199c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:35.290 7f2006199c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 02/03/2025 00:46:35.290 7f2006199c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:35.290 7f2006199c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:35.290 7f2006199c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:46:35.290 7f2006199c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:37.116 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 02/03/2025 00:46:38.240 7f200edb3b08 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table' fixed with properties {'gc_grace_seconds': 864000} 02/03/2025 00:46:38.246 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 02/03/2025 00:46:39.290 7f20061578d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:46:39.291 7f20061578d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:46:39.291 7f20061578d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:46:39.291 7f20061578d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:39.291 7f20061578d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 02/03/2025 00:46:39.291 7f20061578d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:39.291 7f20061578d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:46:39.291 7f20061578d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:46:39.291 7f20061578d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:39.292 7f20061578d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 02/03/2025 00:46:39.292 7f20061578d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:39.292 7f20061578d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:39.292 7f20061578d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:46:39.292 7f20061578d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:40.149 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 02/03/2025 00:46:40.151 7f200edb3b08 [contrail-device-manager] [WARNING]: table 'obj_shared_table', created 02/03/2025 00:46:40.155 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 02/03/2025 00:46:42.220 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 02/03/2025 00:46:43.073 7f200edb3b08 [contrail-device-manager] [WARNING]: table 'obj_shared_table' fixed with properties {'gc_grace_seconds': 864000} 02/03/2025 00:46:43.079 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 02/03/2025 00:46:43.079 7f200704c2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:43.079 7f200704c2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:46:43.079 7f200704c2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:43.092 7f200edb3b08 [contrail-device-manager] [ERROR]: Error in db walk read 'type' 02/03/2025 00:46:43.093 7f200617e780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:43.093 7f200617e780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Zookeeper:Zookeeper[] connection down >>, ] >> 02/03/2025 00:46:43.093 7f200617e780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:43.100 7f200617ea20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:43.101 7f200617ea20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:46:43.101 7f200617ea20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:43.101 7f200edb3b08 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 02/03/2025 00:46:43.101 7f200edb3b08 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 02/03/2025 00:46:43.101 7f200bd32780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 02/03/2025 00:46:43.102 7f200edb3b08 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2672-3. 02/03/2025 00:46:43.106 7f200bd32780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 02/03/2025 00:46:43.248 7f200bd32780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2672-3.'] 02/03/2025 00:46:43.264 7f200bd32780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 02/03/2025 00:46:43.269 7f200bd32780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 02/03/2025 00:46:43.292 7f2006181438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:46:43.293 7f2006181438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:46:43.293 7f2006181438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:46:43.294 7f2006181438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:43.294 7f2006181438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:46:43.294 7f2006181438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:43.294 7f2006181438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:46:43.294 7f2006181438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:46:43.295 7f2006181438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:43.295 7f2006181438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:46:43.295 7f2006181438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:43.295 7f2006181438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:43.295 7f2006181438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:46:43.295 7f2006181438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:43.313 7f200bd32780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2672-3.'] 02/03/2025 00:46:43.613 7f200edb3b08 [contrail-device-manager] [DEBUG]: Fetching all containers 02/03/2025 00:46:43.656 7f200edb3b08 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 02/03/2025 00:46:43.839 7f200edb3b08 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2672-3..config_queue 02/03/2025 00:46:43.839 7f200edb3b08 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2672-3..tftp_queue 02/03/2025 00:46:44.340 7f200edb3b08 [contrail-device-manager] [WARNING]: Process 140 prepared to run in Full mode... 02/03/2025 00:46:44.340 7f200edb3b08 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 02/03/2025 00:46:44.340 7f200edb3b08 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 02/03/2025 00:46:44.343 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page 02/03/2025 00:46:44.343 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 02/03/2025 00:46:44.344 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page 02/03/2025 00:46:44.344 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 02/03/2025 00:46:44.345 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 02/03/2025 00:46:44.345 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 02/03/2025 00:46:44.348 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page 02/03/2025 00:46:44.349 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 02/03/2025 00:46:44.349 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 02/03/2025 00:46:44.349 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 02/03/2025 00:46:44.349 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 02/03/2025 00:46:44.349 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 02/03/2025 00:46:44.349 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 02/03/2025 00:46:44.349 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 02/03/2025 00:46:44.349 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 02/03/2025 00:46:44.349 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 02/03/2025 00:46:44.349 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 02/03/2025 00:46:44.349 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 02/03/2025 00:46:44.349 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 02/03/2025 00:46:44.351 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 02/03/2025 00:46:44.352 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 02/03/2025 00:46:44.352 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 02/03/2025 00:46:44.352 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 02/03/2025 00:46:44.352 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 02/03/2025 00:46:44.352 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 02/03/2025 00:46:44.352 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 02/03/2025 00:46:44.353 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page 02/03/2025 00:46:44.353 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 02/03/2025 00:46:44.353 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 02/03/2025 00:46:44.354 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 02/03/2025 00:46:44.354 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 02/03/2025 00:46:44.354 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 02/03/2025 00:46:44.354 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page 02/03/2025 00:46:44.355 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 02/03/2025 00:46:44.355 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 02/03/2025 00:46:44.355 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 02/03/2025 00:46:44.355 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 02/03/2025 00:46:44.355 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 02/03/2025 00:46:44.355 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 02/03/2025 00:46:44.355 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 02/03/2025 00:46:44.355 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 02/03/2025 00:46:44.355 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 02/03/2025 00:46:44.355 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 02/03/2025 00:46:44.356 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 02/03/2025 00:46:44.356 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 02/03/2025 00:46:44.356 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 02/03/2025 00:46:44.356 7f200edb3b08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 02/03/2025 00:46:44.358 7f200edb3b08 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 02/03/2025 00:46:44.373 7f20058a8048 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 02/03/2025 00:46:44.373 7f20058a8048 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 02/03/2025 00:46:44.428 7f200508f390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:44.429 7f200508f390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 02/03/2025 00:46:44.429 7f200508f390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:44.430 7f200bd32780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 02/03/2025 00:46:44.448 7f2004fd6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:44.448 7f2004fd6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 02/03/2025 00:46:44.448 7f2004fd6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:44.450 7f200bd32780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 02/03/2025 00:46:44.651 7f200bd32780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2672-3.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2672-3..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2672-3..tftp_queue', 'device_manager_ztp.ztp_queue'] 02/03/2025 00:46:44.705 7f200bd32780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 02/03/2025 00:46:44.717 7f200bd32780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 02/03/2025 00:46:44.796 7f200bd32780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2672-3.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2672-3..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2672-3..tftp_queue', 'device_manager_ztp.ztp_queue'] 02/03/2025 00:46:47.294 7f200620ae10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:46:47.295 7f200620ae10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:46:47.296 7f200620ae10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:46:47.296 7f200620ae10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:47.296 7f200620ae10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 02/03/2025 00:46:47.296 7f200620ae10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:47.296 7f200620ae10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:46:47.297 7f200620ae10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:46:47.297 7f200620ae10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:47.297 7f200620ae10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 02/03/2025 00:46:47.297 7f200620ae10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:47.297 7f200620ae10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:47.297 7f200620ae10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:46:47.297 7f200620ae10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:50.590 7f2004f5dcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:50.591 7f2004f5dcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:46:50.591 7f2004f5dcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:51.296 7f2004e7c630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:46:51.297 7f2004e7c630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:46:51.298 7f2004e7c630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:46:51.298 7f2004e7c630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:51.298 7f2004e7c630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:46:51.298 7f2004e7c630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:51.298 7f2004e7c630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:46:51.299 7f2004e7c630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:46:51.299 7f2004e7c630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:51.299 7f2004e7c630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:46:51.299 7f2004e7c630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:51.299 7f2004e7c630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:51.299 7f2004e7c630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:46:51.299 7f2004e7c630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:55.298 7f20061a7f60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:46:55.299 7f20061a7f60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:46:55.299 7f20061a7f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:46:55.299 7f20061a7f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:55.300 7f20061a7f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:46:55.300 7f20061a7f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:55.301 7f20061a7f60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:46:55.301 7f20061a7f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:46:55.301 7f20061a7f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:55.302 7f20061a7f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:46:55.302 7f20061a7f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:55.302 7f20061a7f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:55.302 7f20061a7f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:46:55.302 7f20061a7f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:59.300 7f2006380ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:46:59.301 7f2006380ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:46:59.302 7f2006380ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:46:59.302 7f2006380ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:59.302 7f2006380ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:46:59.302 7f2006380ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:59.302 7f2006380ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:46:59.302 7f2006380ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:46:59.302 7f2006380ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:59.303 7f2006380ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:46:59.303 7f2006380ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:59.303 7f2006380ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:59.303 7f2006380ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:46:59.303 7f2006380ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:03.303 7f2004e816d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:03.304 7f2004e816d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:03.304 7f2004e816d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:03.304 7f2004e816d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:03.304 7f2004e816d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:47:03.304 7f2004e816d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:03.304 7f2004e816d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:03.305 7f2004e816d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:03.305 7f2004e816d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:03.305 7f2004e816d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:03.305 7f2004e816d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:03.305 7f2004e816d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:03.305 7f2004e816d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:03.305 7f2004e816d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:07.304 7f2006223f60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:07.305 7f2006223f60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:07.305 7f2006223f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:07.306 7f2006223f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:07.306 7f2006223f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:47:07.306 7f2006223f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:07.307 7f2006223f60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:07.307 7f2006223f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:07.307 7f2006223f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:07.307 7f2006223f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:07.307 7f2006223f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:07.308 7f2006223f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:07.308 7f2006223f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:07.308 7f2006223f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:11.306 7f2006285198 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:11.307 7f2006285198 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:11.307 7f2006285198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:11.308 7f2006285198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:11.308 7f2006285198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:47:11.308 7f2006285198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:11.308 7f2006285198 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:11.308 7f2006285198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:11.308 7f2006285198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:11.308 7f2006285198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:11.308 7f2006285198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:11.308 7f2006285198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:11.308 7f2006285198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:11.309 7f2006285198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:15.308 7f2006216780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:15.309 7f2006216780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:15.309 7f2006216780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:15.309 7f2006216780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:15.309 7f2006216780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:47:15.309 7f2006216780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:15.309 7f2006216780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:15.310 7f2006216780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:15.310 7f2006216780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:15.310 7f2006216780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:15.310 7f2006216780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:15.310 7f2006216780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:15.310 7f2006216780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:15.310 7f2006216780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:19.309 7f2004e7e780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:19.340 7f2004e7e780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:19.341 7f2004e7e780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:19.341 7f2004e7e780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:19.341 7f2004e7e780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:47:19.341 7f2004e7e780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:19.341 7f2004e7e780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:19.341 7f2004e7e780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:19.341 7f2004e7e780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:19.341 7f2004e7e780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:19.341 7f2004e7e780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:19.341 7f2004e7e780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:19.342 7f2004e7e780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:19.342 7f2004e7e780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:23.341 7f2006216780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:23.342 7f2006216780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:23.343 7f2006216780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:23.343 7f2006216780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:23.343 7f2006216780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:47:23.343 7f2006216780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:23.343 7f2006216780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:23.344 7f2006216780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:23.344 7f2006216780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:23.344 7f2006216780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:23.344 7f2006216780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:23.344 7f2006216780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:23.344 7f2006216780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:23.344 7f2006216780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:27.343 7f2006220828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:27.346 7f2006220828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:27.347 7f2006220828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:27.347 7f2006220828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:27.347 7f2006220828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:47:27.347 7f2006220828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:27.347 7f2006220828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:27.347 7f2006220828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:27.347 7f2006220828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:27.347 7f2006220828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:27.347 7f2006220828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:27.347 7f2006220828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:27.348 7f2006220828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:27.348 7f2006220828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:31.347 7f200bef76d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:31.348 7f200bef76d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:31.348 7f200bef76d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:31.348 7f200bef76d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:31.349 7f200bef76d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:47:31.349 7f200bef76d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:31.349 7f200bef76d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:31.349 7f200bef76d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:31.349 7f200bef76d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:31.349 7f200bef76d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:31.349 7f200bef76d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:31.349 7f200bef76d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:31.349 7f200bef76d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:31.350 7f200bef76d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:35.348 7f2006151e10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:35.351 7f2006151e10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:35.351 7f2006151e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:35.351 7f2006151e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:35.351 7f2006151e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:47:35.351 7f2006151e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:35.351 7f2006151e10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:35.352 7f2006151e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:35.352 7f2006151e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:35.352 7f2006151e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:35.352 7f2006151e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:35.352 7f2006151e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:35.353 7f2006151e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:35.353 7f2006151e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:38.653 7f200edb3b08 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 02/03/2025 00:47:38.653 7f200edb3b08 [contrail-device-manager] [WARNING]: initializing cassandra cluster 02/03/2025 00:47:38.653 7f200edb3b08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 02/03/2025 00:47:38.653 7f200bf4b0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:38.654 7f200bf4b0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 02/03/2025 00:47:38.654 7f200bf4b0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:38.655 7f200edb3b08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 02/03/2025 00:47:38.741 7f200edb3b08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 02/03/2025 00:47:38.743 7f2004fa5a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:38.743 7f2004fa5a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Initializing description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 02/03/2025 00:47:38.743 7f2004fa5a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:38.751 7f200edb3b08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 02/03/2025 00:47:38.751 7f2004fa5828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:38.752 7f2004fa5828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 02/03/2025 00:47:38.752 7f2004fa5828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:38.763 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 02/03/2025 00:47:39.219 7f200edb3b08 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace', created 02/03/2025 00:47:39.223 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 02/03/2025 00:47:39.229 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 02/03/2025 00:47:39.352 7f2004ff7cc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:39.353 7f2004ff7cc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:39.353 7f2004ff7cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:39.353 7f2004ff7cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:39.353 7f2004ff7cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 02/03/2025 00:47:39.353 7f2004ff7cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:39.353 7f2004ff7cc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:39.354 7f2004ff7cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:39.354 7f2004ff7cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:39.354 7f2004ff7cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 02/03/2025 00:47:39.354 7f2004ff7cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:39.354 7f2004ff7cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:39.354 7f2004ff7cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:39.354 7f2004ff7cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:40.123 7f200edb3b08 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None} 02/03/2025 00:47:40.127 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 02/03/2025 00:47:40.128 7f200edb3b08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 02/03/2025 00:47:40.193 7f200edb3b08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 02/03/2025 00:47:40.201 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 02/03/2025 00:47:41.234 7f200edb3b08 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table', created 02/03/2025 00:47:41.239 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 02/03/2025 00:47:41.246 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 02/03/2025 00:47:42.144 7f200edb3b08 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table' fixed with properties {'gc_grace_seconds': 864000} 02/03/2025 00:47:42.149 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 02/03/2025 00:47:42.155 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 02/03/2025 00:47:43.088 7f200edb3b08 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table', created 02/03/2025 00:47:43.095 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 02/03/2025 00:47:43.101 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 02/03/2025 00:47:43.353 7f2004e4dd68 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:43.355 7f2004e4dd68 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:43.355 7f2004e4dd68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:43.355 7f2004e4dd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:43.355 7f2004e4dd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 02/03/2025 00:47:43.355 7f2004e4dd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:43.355 7f2004e4dd68 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:43.355 7f2004e4dd68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:43.355 7f2004e4dd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:43.355 7f2004e4dd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 02/03/2025 00:47:43.356 7f2004e4dd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:43.356 7f2004e4dd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:43.356 7f2004e4dd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:43.356 7f2004e4dd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:45.142 7f200edb3b08 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table' fixed with properties {'gc_grace_seconds': 864000} 02/03/2025 00:47:45.147 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 02/03/2025 00:47:45.155 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 02/03/2025 00:47:46.081 7f200edb3b08 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table', created 02/03/2025 00:47:46.086 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 02/03/2025 00:47:46.094 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 02/03/2025 00:47:47.195 7f200edb3b08 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table' fixed with properties {'gc_grace_seconds': 864000} 02/03/2025 00:47:47.199 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 02/03/2025 00:47:47.205 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 02/03/2025 00:47:47.355 7f2006285240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:47.357 7f2006285240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:47.357 7f2006285240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:47.357 7f2006285240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:47.357 7f2006285240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 02/03/2025 00:47:47.357 7f2006285240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:47.357 7f2006285240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:47.358 7f2006285240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:47.358 7f2006285240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:47.358 7f2006285240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 02/03/2025 00:47:47.358 7f2006285240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:47.358 7f2006285240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:47.358 7f2006285240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:47.358 7f2006285240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:48.234 7f200edb3b08 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table', created 02/03/2025 00:47:48.242 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 02/03/2025 00:47:48.248 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 02/03/2025 00:47:49.147 7f200edb3b08 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table' fixed with properties {'gc_grace_seconds': 864000} 02/03/2025 00:47:49.153 7f200edb3b08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 02/03/2025 00:47:49.153 7f2004fff438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:49.153 7f2004fff438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:49.153 7f2004fff438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:49.195 7f200edb3b08 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.36:5673//', 'pyamqp://guest:guest@10.0.0.32:5673//', 'pyamqp://guest:guest@10.0.0.30:5673//'] 02/03/2025 00:47:49.195 7f200edb3b08 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 02/03/2025 00:47:49.196 7f2004ddd2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:49.196 7f2004ddd2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Initializing description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 02/03/2025 00:47:49.197 7f2004ddd2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:49.197 7f2004ddd2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:49.197 7f2004ddd2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 02/03/2025 00:47:49.197 7f2004ddd2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:49.201 7f200edb3b08 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 02/03/2025 00:47:49.201 7f2004dde588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:49.201 7f2004dde588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:49.201 7f2004dde588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:49.257 7f200edb3b08 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 02/03/2025 00:47:49.258 7f2004e2cc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:49.259 7f2004e2cc18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 02/03/2025 00:47:49.259 7f2004e2cc18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:49.264 7f200edb3b08 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 02/03/2025 00:47:49.264 7f2004deab70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:49.264 7f2004deab70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:49.264 7f2004deab70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:49.265 7f2004deaa20 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 02/03/2025 00:47:49.266 7f2004de4048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:49.266 7f2004de4048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 02/03/2025 00:47:49.266 7f2004de4048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:49.279 7f2004deaa20 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 02/03/2025 00:47:49.279 7f2006155eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:49.279 7f2006155eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:49.279 7f2006155eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:51.358 7f2004dc9438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:51.359 7f2004dc9438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:51.359 7f2004dc9438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:51.359 7f2004dc9438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:51.360 7f2004dc9438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:47:51.360 7f2004dc9438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:51.360 7f2004dc9438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:51.360 7f2004dc9438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:51.360 7f2004dc9438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:51.360 7f2004dc9438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:51.360 7f2004dc9438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:51.360 7f2004dc9438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:51.360 7f2004dc9438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:51.361 7f2004dc9438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:55.360 7f2004dc9438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:55.361 7f2004dc9438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:55.361 7f2004dc9438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:55.361 7f2004dc9438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:55.362 7f2004dc9438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:47:55.362 7f2004dc9438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:55.362 7f2004dc9438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:55.362 7f2004dc9438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:55.362 7f2004dc9438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:55.362 7f2004dc9438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:55.363 7f2004dc9438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:55.363 7f2004dc9438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:55.363 7f2004dc9438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:55.363 7f2004dc9438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:59.363 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:59.364 7f2006195240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:59.364 7f2006195240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:59.364 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:59.365 7f2006195240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:47:59.365 7f2006195240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:59.365 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:59.365 7f2006195240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:59.365 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:59.365 7f2006195240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:59.365 7f2006195240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:59.365 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:59.365 7f2006195240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:59.365 7f2006195240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:03.365 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:03.366 7f2006195240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:03.366 7f2006195240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:03.366 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:03.366 7f2006195240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:03.366 7f2006195240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:03.366 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:03.367 7f2006195240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:03.367 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:03.367 7f2006195240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:03.367 7f2006195240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:03.367 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:03.367 7f2006195240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:03.367 7f2006195240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:07.367 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:07.368 7f2006195240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:07.368 7f2006195240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:07.368 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:07.369 7f2006195240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:07.369 7f2006195240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:07.369 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:07.369 7f2006195240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:07.369 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:07.369 7f2006195240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:07.369 7f2006195240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:07.369 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:07.369 7f2006195240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:07.369 7f2006195240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:11.369 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:11.371 7f2006195240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:11.371 7f2006195240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:11.371 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:11.372 7f2006195240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:11.372 7f2006195240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:11.372 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:11.372 7f2006195240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:11.372 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:11.372 7f2006195240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:11.372 7f2006195240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:11.372 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:11.373 7f2006195240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:11.373 7f2006195240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:15.371 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:15.372 7f2006195240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:15.373 7f2006195240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:15.373 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:15.373 7f2006195240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:15.373 7f2006195240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:15.373 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:15.373 7f2006195240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:15.373 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:15.373 7f2006195240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:15.374 7f2006195240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:15.374 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:15.374 7f2006195240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:15.374 7f2006195240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:19.373 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:19.375 7f2006195240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:19.375 7f2006195240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:19.375 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:19.375 7f2006195240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:19.375 7f2006195240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:19.375 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:19.376 7f2006195240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:19.376 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:19.376 7f2006195240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:19.376 7f2006195240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:19.376 7f2006195240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:19.376 7f2006195240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:19.376 7f2006195240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:21.464 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2672-3'], 'obj_dict': {'bgp_router_parameters': {'address': '10.20.0.14', 'address_families': {'family': ['route-target', 'inet-vpn', 'e-vpn', 'erm-vpn', 'inet6-vpn']}, 'admin_down': False, 'auth_data': None, 'autonomous_system': 64512, 'cluster_id': None, 'gateway_address': None, 'hold_time': 0, 'identifier': '10.20.0.14', 'ipv6_gateway_address': None, 'local_autonomous_system': None, 'port': 179, 'router_type': 'control-node', 'source_port': None, 'vendor': 'contrail'}, 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2672-3', 'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2672-3'], 'id_perms': {'created': '2025-02-03T00:48:21.438830', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-03T00:48:21.438830', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13295998845476960012, 'uuid_mslong': 9654255434009429335}}, 'parent_type': 'routing-instance', 'parent_uuid': '5432d87e-370a-4292-9f7d-35e3dddbdb6d', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '85face2a-b304-4557-b884-e08136a8770c'}, 'oper': 'CREATE', 'request-id': 'req-885f9a06-744c-4c81-8803-6368082171ba', 'type': 'bgp_router', 'uuid': '85face2a-b304-4557-b884-e08136a8770c'} 02/03/2025 00:48:21.540 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2672-2'], 'obj_dict': {'bgp_router_parameters': {'address': '10.20.0.8', 'address_families': {'family': ['route-target', 'inet-vpn', 'e-vpn', 'erm-vpn', 'inet6-vpn']}, 'admin_down': False, 'auth_data': None, 'autonomous_system': 64512, 'cluster_id': None, 'gateway_address': None, 'hold_time': 0, 'identifier': '10.20.0.8', 'ipv6_gateway_address': None, 'local_autonomous_system': None, 'port': 179, 'router_type': 'control-node', 'source_port': None, 'vendor': 'contrail'}, 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2672-2', 'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2672-2'], 'id_perms': {'created': '2025-02-03T00:48:21.531180', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-03T00:48:21.531180', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12380350635648610542, 'uuid_mslong': 4473795269617271503}}, 'parent_type': 'routing-instance', 'parent_uuid': '5432d87e-370a-4292-9f7d-35e3dddbdb6d', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '3e161d18-335c-42cf-abcf-d74f289df0ee'}, 'oper': 'CREATE', 'request-id': 'req-a6ffb72a-cfa4-403b-89f2-c14a42945243', 'type': 'bgp_router', 'uuid': '3e161d18-335c-42cf-abcf-d74f289df0ee'} 02/03/2025 00:48:21.578 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2672-1'], 'obj_dict': {'bgp_router_parameters': {'address': '10.20.0.7', 'address_families': {'family': ['route-target', 'inet-vpn', 'e-vpn', 'erm-vpn', 'inet6-vpn']}, 'admin_down': False, 'auth_data': None, 'autonomous_system': 64512, 'cluster_id': None, 'gateway_address': None, 'hold_time': 0, 'identifier': '10.20.0.7', 'ipv6_gateway_address': None, 'local_autonomous_system': None, 'port': 179, 'router_type': 'control-node', 'source_port': None, 'vendor': 'contrail'}, 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2672-1', 'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2672-1'], 'id_perms': {'created': '2025-02-03T00:48:21.559475', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-03T00:48:21.559475', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12021733272506467936, 'uuid_mslong': 13944970973891150744}}, 'parent_type': 'routing-instance', 'parent_uuid': '5432d87e-370a-4292-9f7d-35e3dddbdb6d', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'c1867d37-95ea-4398-a6d5-c6b7ac3e0660'}, 'oper': 'CREATE', 'request-id': 'req-6b76b4c2-e60a-4f91-9284-bc1bc5a6cf61', 'type': 'bgp_router', 'uuid': 'c1867d37-95ea-4398-a6d5-c6b7ac3e0660'} 02/03/2025 00:48:21.719 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-bb840f44-c828-446a-8fdf-2c01621e7d7a', 'type': 'global_system_config', 'uuid': '4384c55a-db22-4e45-b2e5-b045bd5d7217'} 02/03/2025 00:48:21.810 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-0dec24be-7f11-4807-b9c5-c1177cafbd80', 'type': 'global_system_config', 'uuid': '4384c55a-db22-4e45-b2e5-b045bd5d7217'} 02/03/2025 00:48:21.915 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-abab5af4-1a4d-4410-977b-8901f485a057', 'type': 'global_system_config', 'uuid': '4384c55a-db22-4e45-b2e5-b045bd5d7217'} 02/03/2025 00:48:22.094 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2672-2'], 'oper': 'UPDATE', 'request-id': 'req-e850a680-b594-410b-a8a4-5dfc7020015a', 'type': 'bgp_router', 'uuid': '3e161d18-335c-42cf-abcf-d74f289df0ee'} 02/03/2025 00:48:22.107 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2672-3'], 'oper': 'UPDATE-IMPLICIT', 'request-id': 'req-e850a680-b594-410b-a8a4-5dfc7020015a', 'type': 'bgp_router', 'uuid': '85face2a-b304-4557-b884-e08136a8770c'} 02/03/2025 00:48:22.207 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2672-1'], 'oper': 'UPDATE', 'request-id': 'req-476509b4-e2f2-4e65-9af2-a45f6557c392', 'type': 'bgp_router', 'uuid': 'c1867d37-95ea-4398-a6d5-c6b7ac3e0660'} 02/03/2025 00:48:22.213 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2672-3'], 'oper': 'UPDATE-IMPLICIT', 'request-id': 'req-476509b4-e2f2-4e65-9af2-a45f6557c392', 'type': 'bgp_router', 'uuid': '85face2a-b304-4557-b884-e08136a8770c'} 02/03/2025 00:48:22.246 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2672-1'], 'oper': 'UPDATE', 'request-id': 'req-f872d0e6-a435-4e9c-8995-cf25cc741836', 'type': 'bgp_router', 'uuid': 'c1867d37-95ea-4398-a6d5-c6b7ac3e0660'} 02/03/2025 00:48:22.253 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2672-2'], 'oper': 'UPDATE-IMPLICIT', 'request-id': 'req-f872d0e6-a435-4e9c-8995-cf25cc741836', 'type': 'bgp_router', 'uuid': '3e161d18-335c-42cf-abcf-d74f289df0ee'} 02/03/2025 00:48:23.376 7f2004e842e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:23.377 7f2004e842e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:23.377 7f2004e842e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:23.377 7f2004e842e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:23.378 7f2004e842e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:23.378 7f2004e842e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:23.378 7f2004e842e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:23.378 7f2004e842e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:23.378 7f2004e842e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:23.378 7f2004e842e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:23.378 7f2004e842e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:23.378 7f2004e842e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:23.378 7f2004e842e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:23.378 7f2004e842e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:27.378 7f2004e842e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:27.379 7f2004e842e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:27.379 7f2004e842e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:27.379 7f2004e842e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:27.380 7f2004e842e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:27.380 7f2004e842e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:27.380 7f2004e842e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:27.380 7f2004e842e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:27.380 7f2004e842e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:27.380 7f2004e842e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:27.380 7f2004e842e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:27.380 7f2004e842e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:27.380 7f2004e842e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:27.380 7f2004e842e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:31.380 7f2004e842e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:31.381 7f2004e842e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:31.382 7f2004e842e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:31.382 7f2004e842e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:31.382 7f2004e842e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:31.382 7f2004e842e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:31.382 7f2004e842e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:31.382 7f2004e842e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:31.382 7f2004e842e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:31.382 7f2004e842e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:31.382 7f2004e842e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:31.382 7f2004e842e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:31.383 7f2004e842e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:31.383 7f2004e842e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:35.382 7f2004e842e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:35.384 7f2004e842e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:35.384 7f2004e842e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:35.384 7f2004e842e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:35.384 7f2004e842e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:35.384 7f2004e842e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:35.384 7f2004e842e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:35.385 7f2004e842e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:35.385 7f2004e842e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:35.385 7f2004e842e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:35.385 7f2004e842e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:35.385 7f2004e842e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:35.385 7f2004e842e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:35.385 7f2004e842e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:39.385 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:39.386 7f2004def828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:39.386 7f2004def828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:39.386 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:39.386 7f2004def828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:39.386 7f2004def828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:39.386 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:39.386 7f2004def828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:39.387 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:39.387 7f2004def828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:39.387 7f2004def828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:39.387 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:39.387 7f2004def828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:39.387 7f2004def828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:43.387 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:43.388 7f2004def828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:43.388 7f2004def828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:43.388 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:43.389 7f2004def828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:43.389 7f2004def828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:43.389 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:43.389 7f2004def828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:43.389 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:43.389 7f2004def828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:43.389 7f2004def828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:43.389 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:43.389 7f2004def828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:43.389 7f2004def828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:47.389 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:47.391 7f2004def828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:47.391 7f2004def828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:47.391 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:47.391 7f2004def828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:47.391 7f2004def828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:47.391 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:47.392 7f2004def828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:47.392 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:47.392 7f2004def828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:47.392 7f2004def828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:47.392 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:47.392 7f2004def828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:47.392 7f2004def828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:51.392 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:51.393 7f2004def828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:51.393 7f2004def828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:51.393 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:51.394 7f2004def828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:51.394 7f2004def828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:51.394 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:51.394 7f2004def828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:51.394 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:51.394 7f2004def828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:51.394 7f2004def828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:51.394 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:51.394 7f2004def828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:51.394 7f2004def828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:55.394 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:55.396 7f2004def828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:55.396 7f2004def828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:55.396 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:55.396 7f2004def828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:55.396 7f2004def828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:55.396 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:55.397 7f2004def828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:55.397 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:55.397 7f2004def828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:55.397 7f2004def828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:55.397 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:55.397 7f2004def828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:55.398 7f2004def828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:59.397 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:59.399 7f2004def828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:59.399 7f2004def828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:59.399 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:59.399 7f2004def828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:59.399 7f2004def828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:59.399 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:59.400 7f2004def828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:59.400 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:59.400 7f2004def828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:59.400 7f2004def828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:59.400 7f2004def828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:59.400 7f2004def828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:59.400 7f2004def828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:03.400 7f20070b6cc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:03.401 7f20070b6cc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:03.401 7f20070b6cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:03.401 7f20070b6cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:03.402 7f20070b6cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:03.402 7f20070b6cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:03.402 7f20070b6cc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:03.402 7f20070b6cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:03.402 7f20070b6cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:03.403 7f20070b6cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:03.403 7f20070b6cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:03.403 7f20070b6cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:03.403 7f20070b6cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:03.403 7f20070b6cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:07.402 7f20070b6cc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:07.404 7f20070b6cc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:07.404 7f20070b6cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:07.404 7f20070b6cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:07.404 7f20070b6cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:07.404 7f20070b6cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:07.404 7f20070b6cc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:07.405 7f20070b6cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:07.405 7f20070b6cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:07.405 7f20070b6cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:07.405 7f20070b6cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:07.405 7f20070b6cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:07.405 7f20070b6cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:07.405 7f20070b6cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:11.405 7f20061816d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:11.406 7f20061816d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:11.407 7f20061816d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:11.407 7f20061816d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:11.407 7f20061816d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:11.407 7f20061816d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:11.407 7f20061816d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:11.407 7f20061816d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:11.407 7f20061816d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:11.407 7f20061816d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:11.407 7f20061816d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:11.407 7f20061816d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:11.408 7f20061816d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:11.408 7f20061816d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:15.408 7f20061816d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:15.409 7f20061816d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:15.409 7f20061816d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:15.409 7f20061816d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:15.409 7f20061816d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:15.409 7f20061816d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:15.409 7f20061816d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:15.410 7f20061816d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:15.410 7f20061816d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:15.410 7f20061816d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:15.410 7f20061816d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:15.410 7f20061816d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:15.410 7f20061816d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:15.410 7f20061816d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:19.409 7f20061816d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:19.411 7f20061816d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:19.411 7f20061816d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:19.411 7f20061816d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:19.411 7f20061816d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:19.411 7f20061816d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:19.411 7f20061816d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:19.412 7f20061816d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:19.412 7f20061816d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:19.412 7f20061816d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:19.412 7f20061816d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:19.412 7f20061816d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:19.412 7f20061816d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:19.412 7f20061816d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:20.920 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2672-2.'], 'obj_dict': {'database_node_ip_address': '10.0.0.32', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2672-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2672-2.'], 'id_perms': {'created': '2025-02-03T00:49:20.907213', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-03T00:49:20.907213', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13394647778472387298, 'uuid_mslong': 162119498400220396}}, 'parent_type': 'global-system-config', 'parent_uuid': '4384c55a-db22-4e45-b2e5-b045bd5d7217', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '023ff6d3-294f-44ec-b9e3-592fc75262e2'}, 'oper': 'CREATE', 'request-id': 'req-0b88ec99-100a-4325-8956-007754595bfc', 'type': 'database_node', 'uuid': '023ff6d3-294f-44ec-b9e3-592fc75262e2'} 02/03/2025 00:49:21.655 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2672-3.'], 'obj_dict': {'database_node_ip_address': '10.0.0.30', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2672-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2672-3.'], 'id_perms': {'created': '2025-02-03T00:49:21.642668', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-03T00:49:21.642668', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11710715995185610704, 'uuid_mslong': 11084446157936676816}}, 'parent_type': 'global-system-config', 'parent_uuid': '4384c55a-db22-4e45-b2e5-b045bd5d7217', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '99d3dd0d-7fc2-4fd0-a284-d226ce85ffd0'}, 'oper': 'CREATE', 'request-id': 'req-ac33c51f-1eef-47a6-8b0a-7b3a712496eb', 'type': 'database_node', 'uuid': '99d3dd0d-7fc2-4fd0-a284-d226ce85ffd0'} 02/03/2025 00:49:21.728 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2672-1.'], 'obj_dict': {'database_node_ip_address': '10.0.0.36', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2672-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2672-1.'], 'id_perms': {'created': '2025-02-03T00:49:21.718395', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-03T00:49:21.718395', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11821669104979855486, 'uuid_mslong': 10974466311072399739}}, 'parent_type': 'global-system-config', 'parent_uuid': '4384c55a-db22-4e45-b2e5-b045bd5d7217', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '984d22f6-f6f6-417b-a40f-016ac688c87e'}, 'oper': 'CREATE', 'request-id': 'req-cf21021f-0c1c-4d7b-911e-2f823bed43c8', 'type': 'database_node', 'uuid': '984d22f6-f6f6-417b-a40f-016ac688c87e'} 02/03/2025 00:49:23.412 7f2004e5b630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:23.413 7f2004e5b630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:23.413 7f2004e5b630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:23.413 7f2004e5b630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:23.413 7f2004e5b630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:23.413 7f2004e5b630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:23.413 7f2004e5b630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:23.414 7f2004e5b630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:23.414 7f2004e5b630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:23.414 7f2004e5b630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:23.414 7f2004e5b630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:23.414 7f2004e5b630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:23.414 7f2004e5b630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:23.414 7f2004e5b630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:27.414 7f2004e5b630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:27.416 7f2004e5b630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:27.416 7f2004e5b630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:27.416 7f2004e5b630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:27.416 7f2004e5b630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:27.416 7f2004e5b630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:27.416 7f2004e5b630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:27.417 7f2004e5b630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:27.417 7f2004e5b630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:27.417 7f2004e5b630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:27.417 7f2004e5b630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:27.417 7f2004e5b630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:27.418 7f2004e5b630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:27.418 7f2004e5b630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:31.417 7f2004e5b630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:31.418 7f2004e5b630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:31.418 7f2004e5b630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:31.419 7f2004e5b630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:31.419 7f2004e5b630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:31.419 7f2004e5b630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:31.419 7f2004e5b630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:31.419 7f2004e5b630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:31.419 7f2004e5b630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:31.419 7f2004e5b630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:31.420 7f2004e5b630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:31.420 7f2004e5b630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:31.420 7f2004e5b630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:31.420 7f2004e5b630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:35.420 7f2004e5b630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:35.421 7f2004e5b630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:35.421 7f2004e5b630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:35.421 7f2004e5b630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:35.421 7f2004e5b630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:35.422 7f2004e5b630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:35.422 7f2004e5b630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:35.422 7f2004e5b630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:35.422 7f2004e5b630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:35.422 7f2004e5b630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:35.422 7f2004e5b630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:35.422 7f2004e5b630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:35.422 7f2004e5b630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:35.422 7f2004e5b630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:39.422 7f2004ff7f60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:39.424 7f2004ff7f60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:39.424 7f2004ff7f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:39.424 7f2004ff7f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:39.424 7f2004ff7f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:39.424 7f2004ff7f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:39.424 7f2004ff7f60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:39.424 7f2004ff7f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:39.425 7f2004ff7f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:39.425 7f2004ff7f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:39.425 7f2004ff7f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:39.425 7f2004ff7f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:39.425 7f2004ff7f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:39.425 7f2004ff7f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:43.425 7f2004ff7f60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:43.426 7f2004ff7f60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:43.426 7f2004ff7f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:43.426 7f2004ff7f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:43.426 7f2004ff7f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:43.426 7f2004ff7f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:43.426 7f2004ff7f60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:43.427 7f2004ff7f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:43.427 7f2004ff7f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:43.427 7f2004ff7f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:43.427 7f2004ff7f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:43.427 7f2004ff7f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:43.427 7f2004ff7f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:43.427 7f2004ff7f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:47.426 7f2004ff7f60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:47.428 7f2004ff7f60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:47.428 7f2004ff7f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:47.428 7f2004ff7f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:47.428 7f2004ff7f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:47.428 7f2004ff7f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:47.428 7f2004ff7f60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:47.429 7f2004ff7f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:47.429 7f2004ff7f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:47.429 7f2004ff7f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:47.429 7f2004ff7f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:47.429 7f2004ff7f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:47.429 7f2004ff7f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:47.429 7f2004ff7f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:51.429 7f2004ff7f60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:51.430 7f2004ff7f60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:51.430 7f2004ff7f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:51.430 7f2004ff7f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:51.431 7f2004ff7f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:51.431 7f2004ff7f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:51.431 7f2004ff7f60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:51.431 7f2004ff7f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:51.431 7f2004ff7f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:51.431 7f2004ff7f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:51.431 7f2004ff7f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:51.431 7f2004ff7f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:51.431 7f2004ff7f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:51.431 7f2004ff7f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:55.432 7f2004e7ea20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:55.433 7f2004e7ea20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:55.433 7f2004e7ea20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:55.433 7f2004e7ea20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:55.433 7f2004e7ea20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:55.433 7f2004e7ea20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:55.433 7f2004e7ea20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:55.434 7f2004e7ea20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:55.434 7f2004e7ea20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:55.434 7f2004e7ea20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:55.434 7f2004e7ea20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:55.434 7f2004e7ea20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:55.434 7f2004e7ea20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:55.434 7f2004e7ea20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:59.434 7f200bef78d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:59.435 7f200bef78d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:59.435 7f200bef78d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:59.435 7f200bef78d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:59.436 7f200bef78d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:59.436 7f200bef78d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:59.436 7f200bef78d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:59.436 7f200bef78d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:59.436 7f200bef78d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:59.436 7f200bef78d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:59.436 7f200bef78d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:59.436 7f200bef78d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:59.436 7f200bef78d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:59.437 7f200bef78d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:03.437 7f200bef78d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:50:03.438 7f200bef78d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:50:03.438 7f200bef78d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:50:03.438 7f200bef78d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:50:03.438 7f200bef78d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:50:03.438 7f200bef78d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:50:03.438 7f200bef78d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:50:03.438 7f200bef78d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:50:03.438 7f200bef78d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:03.439 7f200bef78d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:50:03.439 7f200bef78d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:03.439 7f200bef78d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:03.439 7f200bef78d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:50:03.439 7f200bef78d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:07.439 7f200bef78d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:50:07.440 7f200bef78d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:50:07.440 7f200bef78d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:50:07.441 7f200bef78d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:50:07.441 7f200bef78d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:50:07.441 7f200bef78d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:50:07.441 7f200bef78d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:50:07.441 7f200bef78d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:50:07.441 7f200bef78d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:07.441 7f200bef78d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:50:07.441 7f200bef78d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:07.441 7f200bef78d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:07.442 7f200bef78d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:50:07.442 7f200bef78d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:11.441 7f20070bb048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:50:11.444 7f20070bb048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:50:11.444 7f20070bb048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:50:11.444 7f20070bb048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:50:11.445 7f20070bb048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:50:11.445 7f20070bb048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:50:11.445 7f20070bb048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:50:11.445 7f20070bb048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:50:11.445 7f20070bb048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:11.445 7f20070bb048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:50:11.445 7f20070bb048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:11.445 7f20070bb048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:11.445 7f20070bb048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1738543587285289 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.36:8086, 10.0.0.32:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:50:11.446 7f20070bb048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:13.949 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2672-3.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.30', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2672-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2672-3.'], 'id_perms': {'created': '2025-02-03T00:50:13.935716', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-03T00:50:13.935716', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12392599127179602721, 'uuid_mslong': 6352564519686720364}}, 'parent_type': 'global-system-config', 'parent_uuid': '4384c55a-db22-4e45-b2e5-b045bd5d7217', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '5828d7c5-f8ad-4b6c-abfb-5b3f535b2b21'}, 'oper': 'CREATE', 'request-id': 'req-5a999822-6a8e-461e-bf0c-cde707d183e5', 'type': 'analytics_node', 'uuid': '5828d7c5-f8ad-4b6c-abfb-5b3f535b2b21'} 02/03/2025 00:50:14.029 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2672-2.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.32', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2672-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2672-2.'], 'id_perms': {'created': '2025-02-03T00:50:14.015778', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-03T00:50:14.015778', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9718851874195259987, 'uuid_mslong': 145824004982718981}}, 'parent_type': 'global-system-config', 'parent_uuid': '4384c55a-db22-4e45-b2e5-b045bd5d7217', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '02061228-f0b8-4205-86e0-4c49720c4253'}, 'oper': 'CREATE', 'request-id': 'req-6ebf8079-56e6-4eec-aa75-b492d8451f77', 'type': 'analytics_node', 'uuid': '02061228-f0b8-4205-86e0-4c49720c4253'} 02/03/2025 00:50:14.100 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2672-1.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.36', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2672-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2672-1.'], 'id_perms': {'created': '2025-02-03T00:50:14.085933', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-03T00:50:14.085933', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9683408608773198492, 'uuid_mslong': 16280852103200787423}}, 'parent_type': 'global-system-config', 'parent_uuid': '4384c55a-db22-4e45-b2e5-b045bd5d7217', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'e1f134ba-6bf4-4bdf-8662-60d324e2d29c'}, 'oper': 'CREATE', 'request-id': 'req-6f050c70-30bf-40e9-b386-7ea9604b352c', 'type': 'analytics_node', 'uuid': 'e1f134ba-6bf4-4bdf-8662-60d324e2d29c'} 02/03/2025 00:50:15.445 7f20062852e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:50:15.445 7f20062852e8 [contrail-device-manager] [INFO]: Session Event: TCP Connected 02/03/2025 00:50:15.446 7f20062852e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:50:15.446 7f20062852e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:50:15.446 7f20062852e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.36:5673, 10.0.0.32:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.36:9041, 10.0.0.32:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.36:2181, 10.0.0.32:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.36,10.0.0.32,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:50:15.446 7f20062852e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:50:15.446 7f20062852e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/03/2025 00:50:15.446 7f20062852e8 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19 02/03/2025 00:50:15.446 7f20062852e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/03/2025 00:50:15.447 7f20062852e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/03/2025 00:50:15.447 7f20062852e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/03/2025 00:50:15.448 7f2004deeac8 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/03/2025 00:50:15.449 7f2004deeac8 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/03/2025 00:50:27.694 7f2004dee2e8 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001016 02/03/2025 00:50:27.695 7f2004dee2e8 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/03/2025 00:50:27.695 7f2005028cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 02/03/2025 00:50:27.695 7f2005028cc0 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 0 02/03/2025 00:50:27.695 7f2005028cc0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 00:50:27.696 7f2005028cc0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/03/2025 00:50:27.696 7f2005028cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 02/03/2025 00:50:27.696 7f2004deeac8 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/03/2025 00:50:27.697 7f2004deeac8 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/03/2025 00:50:36.060 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2672-1.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.36', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2672-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2672-1.'], 'id_perms': {'created': '2025-02-03T00:50:36.049287', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-03T00:50:36.049287', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13789608789664329655, 'uuid_mslong': 1694521140597113318}}, 'parent_type': 'global-system-config', 'parent_uuid': '4384c55a-db22-4e45-b2e5-b045bd5d7217', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '178425ff-d9ae-41e6-bf5e-88223b8befb7'}, 'oper': 'CREATE', 'request-id': 'req-392da61e-e28b-4292-bd99-a463f0140fd5', 'type': 'analytics_alarm_node', 'uuid': '178425ff-d9ae-41e6-bf5e-88223b8befb7'} 02/03/2025 00:50:36.153 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2672-3.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.30', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2672-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2672-3.'], 'id_perms': {'created': '2025-02-03T00:50:36.118685', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-03T00:50:36.118685', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10011024719154223156, 'uuid_mslong': 8534695136569280972}}, 'parent_type': 'global-system-config', 'parent_uuid': '4384c55a-db22-4e45-b2e5-b045bd5d7217', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '76715402-0977-49cc-8aee-4df0fed79834'}, 'oper': 'CREATE', 'request-id': 'req-16f7976a-dec3-471d-b5cd-1d24ba0cb055', 'type': 'analytics_alarm_node', 'uuid': '76715402-0977-49cc-8aee-4df0fed79834'} 02/03/2025 00:50:36.249 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2672-2.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.32', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2672-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2672-2.'], 'id_perms': {'created': '2025-02-03T00:50:36.221685', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-03T00:50:36.221685', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10918538587547623023, 'uuid_mslong': 17689139797260517797}}, 'parent_type': 'global-system-config', 'parent_uuid': '4384c55a-db22-4e45-b2e5-b045bd5d7217', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'f57c72ec-adf0-41a5-9786-70ff192bde6f'}, 'oper': 'CREATE', 'request-id': 'req-0e3c05a0-2c04-4445-ba0f-f41606b650d7', 'type': 'analytics_alarm_node', 'uuid': 'f57c72ec-adf0-41a5-9786-70ff192bde6f'} 02/03/2025 00:50:37.670 7f2004dee2e8 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001297 02/03/2025 00:50:37.671 7f2004dee2e8 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/03/2025 00:50:37.671 7f200c803828 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 02/03/2025 00:50:37.671 7f200c803828 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 2 02/03/2025 00:50:37.671 7f200c803828 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 00:50:37.672 7f200c803828 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/03/2025 00:50:53.656 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2672-2.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.32', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2672-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2672-2.'], 'id_perms': {'created': '2025-02-03T00:50:53.645623', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-03T00:50:53.645623', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12994325890864831016, 'uuid_mslong': 2929450654443129408}}, 'parent_type': 'global-system-config', 'parent_uuid': '4384c55a-db22-4e45-b2e5-b045bd5d7217', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '28a77fd9-6d1e-4a40-b455-1e8d04a8da28'}, 'oper': 'CREATE', 'request-id': 'req-7011fe91-e494-4900-9ebc-237cf1b8b829', 'type': 'analytics_snmp_node', 'uuid': '28a77fd9-6d1e-4a40-b455-1e8d04a8da28'} 02/03/2025 00:50:53.673 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2672-1.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.36', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2672-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2672-1.'], 'id_perms': {'created': '2025-02-03T00:50:53.661836', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-03T00:50:53.661836', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9400936684142867250, 'uuid_mslong': 17238104422699911916}}, 'parent_type': 'global-system-config', 'parent_uuid': '4384c55a-db22-4e45-b2e5-b045bd5d7217', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'ef3a0ca4-1c1e-4aec-8276-d61b1e463332'}, 'oper': 'CREATE', 'request-id': 'req-9033cca8-bb04-4264-af67-022794a73d1b', 'type': 'analytics_snmp_node', 'uuid': 'ef3a0ca4-1c1e-4aec-8276-d61b1e463332'} 02/03/2025 00:50:53.708 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2672-3.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.30', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2672-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2672-3.'], 'id_perms': {'created': '2025-02-03T00:50:53.695589', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-03T00:50:53.695589', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12462506678997355099, 'uuid_mslong': 11878215179052730443}}, 'parent_type': 'global-system-config', 'parent_uuid': '4384c55a-db22-4e45-b2e5-b045bd5d7217', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'a4d7e5c5-9917-484b-acf3-b7cb9c920a5b'}, 'oper': 'CREATE', 'request-id': 'req-6316c533-0399-41f2-9ec0-ce70b0abee68', 'type': 'analytics_snmp_node', 'uuid': 'a4d7e5c5-9917-484b-acf3-b7cb9c920a5b'} 02/03/2025 00:52:50.746 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-d6b1cd80-4d49-4ed4-919f-853d2b5518dd', 'type': 'global_system_config', 'uuid': '4384c55a-db22-4e45-b2e5-b045bd5d7217'} 02/03/2025 00:52:50.882 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2672-2'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-2672-2', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2672-2'], 'id_perms': {'created': '2025-02-03T00:52:50.871549', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-03T00:52:50.871549', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10384032109055901185, 'uuid_mslong': 17398488161409843211}}, 'parent_type': 'global-system-config', 'parent_uuid': '4384c55a-db22-4e45-b2e5-b045bd5d7217', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'f173d8cd-39ac-400b-901b-7e2fac86fe01', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.23'}, 'oper': 'CREATE', 'request-id': 'req-b1ce10ca-6660-4d36-8aa1-1b29004d62b9', 'type': 'virtual_router', 'uuid': 'f173d8cd-39ac-400b-901b-7e2fac86fe01'} 02/03/2025 00:52:51.227 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2672-2', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2672-2', 'vhost0'], 'id_perms': {'created': '2025-02-03T00:52:51.209001', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-03T00:52:51.209001', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11865935708060288263, 'uuid_mslong': 8591629852849162519}}, 'parent_type': 'virtual-router', 'parent_uuid': 'f173d8cd-39ac-400b-901b-7e2fac86fe01', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '773b99d4-acb7-4917-a4ac-45a8768e8507', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:77:3b:99:d4:ac']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': 'ba069717-392b-446d-8a8b-d13780b83b42'}]}, 'oper': 'CREATE', 'request-id': 'req-3794ecd4-df82-4de1-959b-116b647bb286', 'type': 'virtual_machine_interface', 'uuid': '773b99d4-acb7-4917-a4ac-45a8768e8507'} 02/03/2025 00:52:51.275 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2672-2', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-3794ecd4-df82-4de1-959b-116b647bb286', 'type': 'virtual_machine_interface', 'uuid': '773b99d4-acb7-4917-a4ac-45a8768e8507'} 02/03/2025 00:52:51.809 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-0b8891e3-9610-443b-85ca-c69cc2c5c650', 'type': 'access_control_list', 'uuid': 'cc6129f9-bd61-4b20-b1d2-a0aaf762d3db'} 02/03/2025 00:53:43.076 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2672-1'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-2672-1', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2672-1'], 'id_perms': {'created': '2025-02-03T00:53:42.948752', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-03T00:53:42.948752', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10330400030955246122, 'uuid_mslong': 17543151478289747267}}, 'parent_type': 'global-system-config', 'parent_uuid': '4384c55a-db22-4e45-b2e5-b045bd5d7217', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'f375cb52-6854-4d43-8f5c-f41868c5922a', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.11'}, 'oper': 'CREATE', 'request-id': 'req-6d57fcc4-5ede-4df4-bd38-5be6722d6f42', 'type': 'virtual_router', 'uuid': 'f375cb52-6854-4d43-8f5c-f41868c5922a'} 02/03/2025 00:53:43.424 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2672-1', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2672-1', 'vhost0'], 'id_perms': {'created': '2025-02-03T00:53:43.398997', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-03T00:53:43.398997', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10049016526478239418, 'uuid_mslong': 11587018582035546319}}, 'parent_type': 'virtual-router', 'parent_uuid': 'f375cb52-6854-4d43-8f5c-f41868c5922a', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'a0cd5c02-7c14-40cf-8b75-4749f2c3e6ba', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:a0:cd:5c:02:7c']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': 'ba069717-392b-446d-8a8b-d13780b83b42'}]}, 'oper': 'CREATE', 'request-id': 'req-86205e21-6bb2-4e78-9a5f-0aa2b2243a7f', 'type': 'virtual_machine_interface', 'uuid': 'a0cd5c02-7c14-40cf-8b75-4749f2c3e6ba'} 02/03/2025 00:53:43.474 7f2004dea198 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2672-1', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-86205e21-6bb2-4e78-9a5f-0aa2b2243a7f', 'type': 'virtual_machine_interface', 'uuid': 'a0cd5c02-7c14-40cf-8b75-4749f2c3e6ba'} 02/03/2025 00:53:58.806 7f2004ff7588 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 00:54:23.192 7f2004ff7588 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 00:54:38.563 7f2004ffb588 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 00:54:54.114 7f2006199eb8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 00:55:09.358 7f200703d6d8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 00:55:24.665 7f200703d6d8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 00:55:40.278 7f2004de3b70 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 00:55:55.592 7f2004de3b70 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 00:56:10.927 7f2004dc9d68 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 00:56:26.421 7f2004dc9d68 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 00:56:41.639 7f2004e3eb70 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 00:56:56.724 7f2004e3eb70 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 00:57:12.167 7f2004dee588 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 00:57:28.320 7f2004dc9e10 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 00:57:43.673 7f20070b6d68 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 00:57:59.144 7f20070b6cc0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 00:58:14.206 7f200cc936d8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 00:58:29.618 7f2004ddd390 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 00:58:44.869 7f2004ff78d0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 00:59:00.449 7f2004ff78d0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 00:59:15.433 7f2004dddd68 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 00:59:30.418 7f2004dee630 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 00:59:45.599 7f2004e51240 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:00:01.035 7f200bdf18d0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:00:16.274 7f200bd54240 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:00:31.505 7f2004de3630 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:00:46.934 7f2004deecc0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:01:02.375 7f2004ffb588 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:01:17.389 7f2014cad978 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:01:32.969 7f2014cad978 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:01:48.542 7f20070ae6d8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:02:03.826 7f200650a8d0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:02:19.000 7f2005028cc0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:02:34.464 7f2004deeb70 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:02:50.073 7f2004dc9d68 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:03:05.600 7f20070b6cc0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:03:20.692 7f2004ff7198 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:03:36.853 7f2004ddd390 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:03:51.876 7f20070a7438 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:04:07.186 7f20070a7438 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:04:22.373 7f2004de36d8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:04:37.920 7f2004e3e240 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:04:53.335 7f20062852e8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:05:08.559 7f20050284e0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:05:24.961 7f2004ff7f60 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:05:40.326 7f2004de3630 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:05:55.302 7f2004de3630 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:06:10.510 7f2004e51588 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:06:26.405 7f2004e51588 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:06:41.835 7f2004de3b70 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:06:57.251 7f2004de3b70 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:07:12.680 7f2004e2c978 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:07:27.901 7f2004e842e8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:07:43.489 7f2004dee588 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:07:59.099 7f2004ddd198 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:08:14.825 7f2004e3e630 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:08:30.495 7f2004dc9d68 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:08:45.937 7f200bdf1780 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:09:01.740 7f200bdf1780 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:09:17.272 7f2004ddd2e8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:09:32.818 7f2004deea20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:09:37.698 7f2004deea20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:09:42.464 7f20070b6cc0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/03/2025 01:10:19.151 7f200cc936d8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map