07/10/2025 00:41:43.501 7f4be223a188 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 07/10/2025 00:41:43.502 7f4be223a188 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 07/10/2025 00:41:43.502 7f4be223a188 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 07/10/2025 00:41:43.502 7f4bd99312e8 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 07/10/2025 00:41:43.502 7f4bd99312e8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 07/10/2025 00:41:43.503 7f4bd99312e8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 07/10/2025 00:41:43.503 7f4bd99312e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 07/10/2025 00:41:43.503 7f4bd99312e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:43.503 7f4bd99312e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 07/10/2025 00:41:43.503 7f4bd99312e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:43.503 7f4bd99312e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:43.503 7f4bd99312e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 start_time = 1752108103502875 collector_name = collector_ip = collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 07/10/2025 00:41:43.504 7f4bd99312e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:43.547 7f4be223a188 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.41:5673//', 'pyamqp://guest:guest@10.0.0.61:5673//', 'pyamqp://guest:guest@10.0.0.130:5673//'] 07/10/2025 00:41:43.556 7f4be223a188 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 07/10/2025 00:41:43.556 7f4bd9884588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:43.557 7f4bd9884588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 07/10/2025 00:41:43.557 7f4bd9884588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:43.557 7f4bd9884588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:43.557 7f4bd9884588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 07/10/2025 00:41:43.557 7f4bd9884588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:43.561 7f4be223a188 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 07/10/2025 00:41:43.561 7f4bd9884240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:43.561 7f4bd9884240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 07/10/2025 00:41:43.561 7f4bd9884240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:43.575 7f4be223a188 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 07/10/2025 00:41:43.576 7f4bd98840f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:43.576 7f4bd98840f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 07/10/2025 00:41:43.576 7f4bd98840f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:43.580 7f4be223a188 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 07/10/2025 00:41:43.580 7f4be8135978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:43.580 7f4be8135978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 07/10/2025 00:41:43.580 7f4be8135978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:43.595 7f4be223a188 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 07/10/2025 00:41:43.595 7f4be223a188 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 07/10/2025 00:41:43.595 7f4be223a188 [contrail-device-manager] [WARNING]: initializing cassandra cluster 07/10/2025 00:41:43.595 7f4be223a188 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 07/10/2025 00:41:43.596 7f4bdf1bb6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 07/10/2025 00:41:43.596 7f4bdf1bb6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 07/10/2025 00:41:43.596 7f4bdc0d0978 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 07/10/2025 00:41:43.596 7f4bdc0d0978 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 07/10/2025 00:41:43.596 7f4bd971d588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:43.597 7f4bd971d588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 07/10/2025 00:41:43.597 7f4bd971d588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:43.598 7f4be223a188 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 07/10/2025 00:41:43.603 7f4bdc0d0978 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 07/10/2025 00:41:43.603 7f4bdf1bb6d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 07/10/2025 00:41:43.603 7f4bdf1bb6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 07/10/2025 00:41:43.660 7f4be223a188 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 07/10/2025 00:41:43.661 7f4bd96bc240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:43.662 7f4bd96bc240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 07/10/2025 00:41:43.662 7f4bd96bc240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:43.669 7f4be223a188 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 07/10/2025 00:41:43.669 7f4bd96b8588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:43.670 7f4bd96b8588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 07/10/2025 00:41:43.670 7f4bd96b8588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:44.682 7f4be223a188 [contrail-device-manager] [WARNING]: Waiting for keyspace config_db_uuid to be created 07/10/2025 00:41:44.705 7f4be223a188 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 07/10/2025 00:41:44.734 7f4be223a188 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 07/10/2025 00:41:46.682 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/10/2025 00:41:47.503 7f4bd8e1e438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:41:47.504 7f4bd8e1e438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:41:47.504 7f4bd8e1e438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:41:47.504 7f4bd8e1e438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:41:47.504 7f4bd8e1e438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 07/10/2025 00:41:47.505 7f4bd8e1e438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:41:47.505 7f4bd8e1e438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:41:47.505 7f4bd8e1e438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:41:47.505 7f4bd8e1e438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:47.505 7f4bd8e1e438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 07/10/2025 00:41:47.505 7f4bd8e1e438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:47.505 7f4bd8e1e438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:47.505 7f4bd8e1e438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:41:47.505 7f4bd8e1e438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:47.680 7f4be223a188 [contrail-device-manager] [WARNING]: table 'obj_uuid_table', created 07/10/2025 00:41:47.685 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/10/2025 00:41:51.504 7f4bd8de8ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:41:51.506 7f4bd8de8ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:41:51.506 7f4bd8de8ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:41:51.506 7f4bd8de8ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:41:51.507 7f4bd8de8ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 07/10/2025 00:41:51.507 7f4bd8de8ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:41:51.507 7f4bd8de8ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:41:51.507 7f4bd8de8ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:41:51.507 7f4bd8de8ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:51.507 7f4bd8de8ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 07/10/2025 00:41:51.507 7f4bd8de8ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:51.507 7f4bd8de8ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:51.507 7f4bd8de8ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:41:51.507 7f4bd8de8ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:51.994 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/10/2025 00:41:53.661 7f4be223a188 [contrail-device-manager] [WARNING]: table 'obj_uuid_table' fixed with properties {'gc_grace_seconds': 864000} 07/10/2025 00:41:53.667 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/10/2025 00:41:55.507 7f4bd8e042e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:41:55.508 7f4bd8e042e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:41:55.508 7f4bd8e042e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:41:55.508 7f4bd8e042e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:41:55.508 7f4bd8e042e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 07/10/2025 00:41:55.508 7f4bd8e042e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:41:55.508 7f4bd8e042e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:41:55.509 7f4bd8e042e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:41:55.509 7f4bd8e042e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:55.509 7f4bd8e042e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 07/10/2025 00:41:55.509 7f4bd8e042e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:55.509 7f4bd8e042e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:55.509 7f4bd8e042e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:41:55.509 7f4bd8e042e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:56.754 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/10/2025 00:41:57.724 7f4be223a188 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table', created 07/10/2025 00:41:57.731 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/10/2025 00:41:58.811 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/10/2025 00:41:59.509 7f4bd8dbf978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:41:59.510 7f4bd8dbf978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:41:59.510 7f4bd8dbf978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:41:59.510 7f4bd8dbf978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:41:59.510 7f4bd8dbf978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 07/10/2025 00:41:59.510 7f4bd8dbf978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:41:59.511 7f4bd8dbf978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:41:59.511 7f4bd8dbf978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:41:59.511 7f4bd8dbf978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:59.511 7f4bd8dbf978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 07/10/2025 00:41:59.511 7f4bd8dbf978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:59.511 7f4bd8dbf978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:59.511 7f4bd8dbf978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:41:59.511 7f4bd8dbf978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:41:59.634 7f4be223a188 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table' fixed with properties {'gc_grace_seconds': 864000} 07/10/2025 00:41:59.639 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/10/2025 00:42:01.852 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/10/2025 00:42:03.017 7f4be223a188 [contrail-device-manager] [WARNING]: table 'obj_shared_table', created 07/10/2025 00:42:03.022 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/10/2025 00:42:03.510 7f4bd8de8588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:42:03.511 7f4bd8de8588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:42:03.511 7f4bd8de8588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:42:03.512 7f4bd8de8588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:03.512 7f4bd8de8588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 07/10/2025 00:42:03.512 7f4bd8de8588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:03.512 7f4bd8de8588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:42:03.512 7f4bd8de8588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:42:03.512 7f4bd8de8588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:03.512 7f4bd8de8588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 07/10/2025 00:42:03.512 7f4bd8de8588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:03.512 7f4bd8de8588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:03.513 7f4bd8de8588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:42:03.513 7f4bd8de8588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:04.796 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/10/2025 00:42:06.105 7f4be223a188 [contrail-device-manager] [WARNING]: table 'obj_shared_table' fixed with properties {'gc_grace_seconds': 864000} 07/10/2025 00:42:06.111 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/10/2025 00:42:06.112 7f4bd8e0aa20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:06.112 7f4bd8e0aa20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:42:06.112 7f4bd8e0aa20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:06.124 7f4be223a188 [contrail-device-manager] [ERROR]: Error in db walk read 'type' 07/10/2025 00:42:06.126 7f4bd8e04eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:06.126 7f4bd8e04eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Zookeeper:Zookeeper[] connection down >>, ] >> 07/10/2025 00:42:06.126 7f4bd8e04eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:06.137 7f4bd8de88d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:06.137 7f4bd8de88d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:42:06.138 7f4bd8de88d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:06.138 7f4be223a188 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 07/10/2025 00:42:06.138 7f4be223a188 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 07/10/2025 00:42:06.139 7f4bdf1bb6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 07/10/2025 00:42:06.140 7f4be223a188 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3621-2. 07/10/2025 00:42:06.145 7f4bdf1bb6d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 07/10/2025 00:42:06.316 7f4bdf1bb6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3621-2.'] 07/10/2025 00:42:06.345 7f4bdf1bb6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 07/10/2025 00:42:06.351 7f4bdf1bb6d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 07/10/2025 00:42:06.391 7f4bdf1bb6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3621-2.'] 07/10/2025 00:42:06.654 7f4be223a188 [contrail-device-manager] [DEBUG]: Fetching all containers 07/10/2025 00:42:06.697 7f4be223a188 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 07/10/2025 00:42:06.883 7f4be223a188 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3621-2..config_queue 07/10/2025 00:42:06.883 7f4be223a188 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3621-2..tftp_queue 07/10/2025 00:42:07.384 7f4be223a188 [contrail-device-manager] [WARNING]: Process 137 prepared to run in Full mode... 07/10/2025 00:42:07.384 7f4be223a188 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 07/10/2025 00:42:07.384 7f4be223a188 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 07/10/2025 00:42:07.387 7f4be223a188 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page 07/10/2025 00:42:07.387 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 07/10/2025 00:42:07.388 7f4be223a188 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page 07/10/2025 00:42:07.388 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 07/10/2025 00:42:07.389 7f4be223a188 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 07/10/2025 00:42:07.389 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 07/10/2025 00:42:07.392 7f4be223a188 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page 07/10/2025 00:42:07.392 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 07/10/2025 00:42:07.392 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 07/10/2025 00:42:07.392 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 07/10/2025 00:42:07.392 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 07/10/2025 00:42:07.392 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 07/10/2025 00:42:07.393 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 07/10/2025 00:42:07.393 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 07/10/2025 00:42:07.393 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 07/10/2025 00:42:07.393 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 07/10/2025 00:42:07.393 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 07/10/2025 00:42:07.393 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 07/10/2025 00:42:07.393 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 07/10/2025 00:42:07.394 7f4be223a188 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 07/10/2025 00:42:07.394 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 07/10/2025 00:42:07.394 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 07/10/2025 00:42:07.394 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 07/10/2025 00:42:07.395 7f4be223a188 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 07/10/2025 00:42:07.395 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 07/10/2025 00:42:07.395 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 07/10/2025 00:42:07.395 7f4be223a188 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page 07/10/2025 00:42:07.395 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 07/10/2025 00:42:07.395 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 07/10/2025 00:42:07.396 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 07/10/2025 00:42:07.396 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 07/10/2025 00:42:07.396 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 07/10/2025 00:42:07.396 7f4be223a188 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page 07/10/2025 00:42:07.397 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 07/10/2025 00:42:07.397 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 07/10/2025 00:42:07.397 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 07/10/2025 00:42:07.397 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 07/10/2025 00:42:07.397 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 07/10/2025 00:42:07.397 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 07/10/2025 00:42:07.397 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 07/10/2025 00:42:07.397 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 07/10/2025 00:42:07.397 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 07/10/2025 00:42:07.397 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 07/10/2025 00:42:07.397 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 07/10/2025 00:42:07.397 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 07/10/2025 00:42:07.398 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 07/10/2025 00:42:07.398 7f4be223a188 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 07/10/2025 00:42:07.402 7f4be223a188 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 07/10/2025 00:42:07.458 7f4bd8e13eb8 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 07/10/2025 00:42:07.459 7f4bd8e13eb8 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 07/10/2025 00:42:07.459 7f4bd84d4438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:07.459 7f4bd84d4438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 07/10/2025 00:42:07.459 7f4bd84d4438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:07.470 7f4bd842b390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:07.470 7f4bd842b390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 07/10/2025 00:42:07.470 7f4bd842b390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:07.477 7f4bdf1bb6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 07/10/2025 00:42:07.482 7f4bdf1bb6d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 07/10/2025 00:42:07.512 7f4bd842bb70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:42:07.513 7f4bd842bb70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:42:07.513 7f4bd842bb70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:42:07.513 7f4bd842bb70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:07.513 7f4bd842bb70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 07/10/2025 00:42:07.513 7f4bd842bb70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:07.513 7f4bd842bb70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:42:07.513 7f4bd842bb70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:42:07.513 7f4bd842bb70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:07.514 7f4bd842bb70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 07/10/2025 00:42:07.514 7f4bd842bb70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:07.514 7f4bd842bb70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:07.514 7f4bd842bb70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:42:07.514 7f4bd842bb70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:07.652 7f4bdf1bb6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3621-2.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3621-2..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3621-2..tftp_queue', 'device_manager_ztp.ztp_queue'] 07/10/2025 00:42:11.514 7f4bd84090f0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:42:11.516 7f4bd84090f0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:42:11.516 7f4bd84090f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:42:11.516 7f4bd84090f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:11.516 7f4bd84090f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 07/10/2025 00:42:11.516 7f4bd84090f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:11.516 7f4bd84090f0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:42:11.517 7f4bd84090f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:42:11.517 7f4bd84090f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:11.517 7f4bd84090f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 07/10/2025 00:42:11.517 7f4bd84090f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:11.517 7f4bd84090f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:11.517 7f4bd84090f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:42:11.517 7f4bd84090f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:15.517 7f4bd835dc18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:42:15.518 7f4bd835dc18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:42:15.518 7f4bd835dc18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:42:15.518 7f4bd835dc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:15.518 7f4bd835dc18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 07/10/2025 00:42:15.518 7f4bd835dc18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:15.518 7f4bd835dc18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:42:15.519 7f4bd835dc18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:42:15.519 7f4bd835dc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:15.519 7f4bd835dc18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 07/10/2025 00:42:15.519 7f4bd835dc18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:15.519 7f4bd835dc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:15.519 7f4bd835dc18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:42:15.519 7f4bd835dc18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:16.577 7f4bd840c828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:16.578 7f4bd840c828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:42:16.578 7f4bd840c828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:19.518 7f4bd8dada20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:42:19.520 7f4bd8dada20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:42:19.520 7f4bd8dada20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:42:19.520 7f4bd8dada20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:19.520 7f4bd8dada20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/10/2025 00:42:19.520 7f4bd8dada20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:19.520 7f4bd8dada20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:42:19.520 7f4bd8dada20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:42:19.521 7f4bd8dada20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:19.521 7f4bd8dada20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:42:19.521 7f4bd8dada20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:19.521 7f4bd8dada20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:19.521 7f4bd8dada20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:42:19.521 7f4bd8dada20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:23.521 7f4bd8e2e8d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:42:23.522 7f4bd8e2e8d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:42:23.522 7f4bd8e2e8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:42:23.522 7f4bd8e2e8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:23.522 7f4bd8e2e8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/10/2025 00:42:23.522 7f4bd8e2e8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:23.522 7f4bd8e2e8d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:42:23.523 7f4bd8e2e8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:42:23.523 7f4bd8e2e8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:23.523 7f4bd8e2e8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:42:23.523 7f4bd8e2e8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:23.523 7f4bd8e2e8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:23.523 7f4bd8e2e8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:42:23.523 7f4bd8e2e8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:27.524 7f4bd8e8c6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:42:27.525 7f4bd8e8c6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:42:27.525 7f4bd8e8c6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:42:27.525 7f4bd8e8c6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:27.525 7f4bd8e8c6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/10/2025 00:42:27.525 7f4bd8e8c6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:27.525 7f4bd8e8c6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:42:27.526 7f4bd8e8c6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:42:27.526 7f4bd8e8c6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:27.526 7f4bd8e8c6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:42:27.526 7f4bd8e8c6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:27.526 7f4bd8e8c6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:27.526 7f4bd8e8c6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:42:27.526 7f4bd8e8c6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:31.527 7f4bd8e8c6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:42:31.528 7f4bd8e8c6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:42:31.529 7f4bd8e8c6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:42:31.529 7f4bd8e8c6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:31.529 7f4bd8e8c6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/10/2025 00:42:31.529 7f4bd8e8c6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:31.529 7f4bd8e8c6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:42:31.529 7f4bd8e8c6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:42:31.529 7f4bd8e8c6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:31.529 7f4bd8e8c6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:42:31.530 7f4bd8e8c6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:31.530 7f4bd8e8c6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:31.530 7f4bd8e8c6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:42:31.530 7f4bd8e8c6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:35.529 7f4bd8da5390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:42:35.532 7f4bd8da5390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:42:35.532 7f4bd8da5390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:42:35.532 7f4bd8da5390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:35.532 7f4bd8da5390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/10/2025 00:42:35.532 7f4bd8da5390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:35.532 7f4bd8da5390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:42:35.533 7f4bd8da5390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:42:35.533 7f4bd8da5390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:35.533 7f4bd8da5390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:42:35.533 7f4bd8da5390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:35.533 7f4bd8da5390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:35.533 7f4bd8da5390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:42:35.533 7f4bd8da5390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:39.533 7f4bd8da5390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:42:39.534 7f4bd8da5390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:42:39.535 7f4bd8da5390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:42:39.535 7f4bd8da5390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:39.535 7f4bd8da5390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/10/2025 00:42:39.535 7f4bd8da5390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:39.535 7f4bd8da5390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:42:39.536 7f4bd8da5390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:42:39.536 7f4bd8da5390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:39.536 7f4bd8da5390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:42:39.536 7f4bd8da5390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:39.536 7f4bd8da5390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:39.536 7f4bd8da5390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:42:39.536 7f4bd8da5390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:43.536 7f4bd8da5390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:42:43.538 7f4bd8da5390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:42:43.538 7f4bd8da5390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:42:43.538 7f4bd8da5390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:43.538 7f4bd8da5390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/10/2025 00:42:43.538 7f4bd8da5390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:43.538 7f4bd8da5390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:42:43.539 7f4bd8da5390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:42:43.539 7f4bd8da5390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:43.539 7f4bd8da5390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:42:43.539 7f4bd8da5390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:43.539 7f4bd8da5390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:43.539 7f4bd8da5390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:42:43.539 7f4bd8da5390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:47.539 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:42:47.541 7f4bd8dc9b70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:42:47.541 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:42:47.541 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:47.541 7f4bd8dc9b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/10/2025 00:42:47.541 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:47.541 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:42:47.542 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:42:47.542 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:47.542 7f4bd8dc9b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:42:47.542 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:47.542 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:47.542 7f4bd8dc9b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:42:47.542 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:51.542 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:42:51.544 7f4bd8dc9b70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:42:51.544 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:42:51.544 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:51.544 7f4bd8dc9b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/10/2025 00:42:51.544 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:51.545 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:42:51.545 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:42:51.545 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:51.545 7f4bd8dc9b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:42:51.545 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:51.545 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:51.546 7f4bd8dc9b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:42:51.546 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:55.544 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:42:55.545 7f4bd8dc9b70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:42:55.546 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:42:55.546 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:55.546 7f4bd8dc9b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/10/2025 00:42:55.546 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:55.546 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:42:55.546 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:42:55.546 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:55.546 7f4bd8dc9b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:42:55.546 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:55.546 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:55.547 7f4bd8dc9b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:42:55.547 7f4bd8dc9b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:59.546 7f4bd8e24630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:42:59.547 7f4bd8e24630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:42:59.547 7f4bd8e24630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:42:59.547 7f4bd8e24630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:59.547 7f4bd8e24630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/10/2025 00:42:59.548 7f4bd8e24630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:42:59.548 7f4bd8e24630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:42:59.548 7f4bd8e24630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:42:59.548 7f4bd8e24630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:59.548 7f4bd8e24630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:42:59.548 7f4bd8e24630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:59.548 7f4bd8e24630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:42:59.548 7f4bd8e24630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:42:59.548 7f4bd8e24630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:03.548 7f4bd8dd5828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:43:03.548 7f4bd8dd5828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:43:03.549 7f4bd8dd5828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:43:03.549 7f4bd8dd5828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:03.549 7f4bd8dd5828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/10/2025 00:43:03.549 7f4bd8dd5828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:03.549 7f4bd8dd5828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:43:03.549 7f4bd8dd5828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:43:03.549 7f4bd8dd5828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:03.549 7f4bd8dd5828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:43:03.549 7f4bd8dd5828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:03.550 7f4bd8dd5828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:03.550 7f4bd8dd5828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:43:03.550 7f4bd8dd5828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:07.550 7f4bd8476978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:43:07.551 7f4bd8476978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:43:07.551 7f4bd8476978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:43:07.551 7f4bd8476978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:07.552 7f4bd8476978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/10/2025 00:43:07.552 7f4bd8476978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:07.552 7f4bd8476978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:43:07.553 7f4bd8476978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:43:07.553 7f4bd8476978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:07.553 7f4bd8476978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:43:07.553 7f4bd8476978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:07.553 7f4bd8476978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:07.553 7f4bd8476978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:43:07.553 7f4bd8476978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:11.551 7f4bd8476978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:43:11.553 7f4bd8476978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:43:11.553 7f4bd8476978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:43:11.553 7f4bd8476978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:11.553 7f4bd8476978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/10/2025 00:43:11.553 7f4bd8476978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:11.553 7f4bd8476978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:43:11.553 7f4bd8476978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:43:11.553 7f4bd8476978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:11.554 7f4bd8476978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:43:11.554 7f4bd8476978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:11.554 7f4bd8476978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:11.554 7f4bd8476978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:43:11.554 7f4bd8476978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:15.553 7f4bdf406828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:43:15.554 7f4bdf406828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:43:15.554 7f4bdf406828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:43:15.554 7f4bdf406828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:15.555 7f4bdf406828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/10/2025 00:43:15.555 7f4bdf406828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:15.555 7f4bdf406828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:43:15.555 7f4bdf406828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:43:15.555 7f4bdf406828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:15.555 7f4bdf406828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:43:15.555 7f4bdf406828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:15.555 7f4bdf406828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:15.555 7f4bdf406828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:43:15.555 7f4bdf406828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:19.555 7f4bd8ddf0f0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:43:19.556 7f4bd8ddf0f0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:43:19.556 7f4bd8ddf0f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:43:19.557 7f4bd8ddf0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:19.557 7f4bd8ddf0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/10/2025 00:43:19.557 7f4bd8ddf0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:19.557 7f4bd8ddf0f0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:43:19.557 7f4bd8ddf0f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:43:19.557 7f4bd8ddf0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:19.558 7f4bd8ddf0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:43:19.558 7f4bd8ddf0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:19.558 7f4bd8ddf0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:19.558 7f4bd8ddf0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:43:19.558 7f4bd8ddf0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:23.557 7f4bd8daf048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:43:23.558 7f4bd8daf048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:43:23.559 7f4bd8daf048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:43:23.559 7f4bd8daf048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:23.559 7f4bd8daf048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/10/2025 00:43:23.559 7f4bd8daf048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:23.559 7f4bd8daf048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:43:23.559 7f4bd8daf048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:43:23.559 7f4bd8daf048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:23.559 7f4bd8daf048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:43:23.559 7f4bd8daf048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:23.559 7f4bd8daf048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:23.560 7f4bd8daf048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:43:23.560 7f4bd8daf048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:27.560 7f4bd82758d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:43:27.561 7f4bd82758d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:43:27.561 7f4bd82758d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:43:27.561 7f4bd82758d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:27.561 7f4bd82758d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/10/2025 00:43:27.561 7f4bd82758d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:27.562 7f4bd82758d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:43:27.562 7f4bd82758d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:43:27.562 7f4bd82758d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:27.562 7f4bd82758d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:43:27.562 7f4bd82758d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:27.562 7f4bd82758d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:27.563 7f4bd82758d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:43:27.563 7f4bd82758d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:31.562 7f4bd8275828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:43:31.563 7f4bd8275828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:43:31.563 7f4bd8275828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:43:31.563 7f4bd8275828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:31.564 7f4bd8275828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/10/2025 00:43:31.564 7f4bd8275828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:31.564 7f4bd8275828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:43:31.564 7f4bd8275828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:43:31.564 7f4bd8275828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:31.564 7f4bd8275828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:43:31.564 7f4bd8275828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:31.564 7f4bd8275828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:31.564 7f4bd8275828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:43:31.564 7f4bd8275828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:35.563 7f4bdf921a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:43:35.570 7f4bdf921a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:43:35.570 7f4bdf921a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:43:35.570 7f4bdf921a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:35.570 7f4bdf921a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/10/2025 00:43:35.570 7f4bdf921a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:35.570 7f4bdf921a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:43:35.571 7f4bdf921a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:43:35.571 7f4bdf921a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:35.572 7f4bdf921a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:43:35.572 7f4bdf921a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:35.572 7f4bdf921a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:35.572 7f4bdf921a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:43:35.572 7f4bdf921a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:37.116 7f4be223a188 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 07/10/2025 00:43:37.116 7f4be223a188 [contrail-device-manager] [WARNING]: initializing cassandra cluster 07/10/2025 00:43:37.117 7f4be223a188 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 07/10/2025 00:43:37.117 7f4bd8400780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:37.117 7f4bd8400780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 07/10/2025 00:43:37.117 7f4bd8400780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:37.118 7f4be223a188 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 07/10/2025 00:43:37.160 7f4be223a188 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 07/10/2025 00:43:37.162 7f4bd83eeac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:37.162 7f4bd83eeac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Initializing description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 07/10/2025 00:43:37.162 7f4bd83eeac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:37.168 7f4be223a188 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 07/10/2025 00:43:37.169 7f4bd83ee978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:37.169 7f4bd83ee978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 07/10/2025 00:43:37.169 7f4bd83ee978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:37.199 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/10/2025 00:43:38.092 7f4be223a188 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace', created 07/10/2025 00:43:38.099 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/10/2025 00:43:38.106 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/10/2025 00:43:38.970 7f4be223a188 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None} 07/10/2025 00:43:38.974 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/10/2025 00:43:38.974 7f4be223a188 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 07/10/2025 00:43:39.057 7f4be223a188 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 07/10/2025 00:43:39.066 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/10/2025 00:43:39.570 7f4bd83eeac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:43:39.571 7f4bd83eeac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:43:39.571 7f4bd83eeac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:43:39.572 7f4bd83eeac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:39.572 7f4bd83eeac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 07/10/2025 00:43:39.572 7f4bd83eeac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:39.572 7f4bd83eeac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:43:39.573 7f4bd83eeac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:43:39.573 7f4bd83eeac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:39.573 7f4bd83eeac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 07/10/2025 00:43:39.573 7f4bd83eeac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:39.573 7f4bd83eeac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:39.573 7f4bd83eeac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:43:39.573 7f4bd83eeac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:39.800 7f4be223a188 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table', created 07/10/2025 00:43:39.805 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/10/2025 00:43:39.810 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/10/2025 00:43:40.719 7f4be223a188 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table' fixed with properties {'gc_grace_seconds': 864000} 07/10/2025 00:43:40.724 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/10/2025 00:43:40.729 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/10/2025 00:43:41.641 7f4be223a188 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table', created 07/10/2025 00:43:41.645 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/10/2025 00:43:41.655 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/10/2025 00:43:43.146 7f4be223a188 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table' fixed with properties {'gc_grace_seconds': 864000} 07/10/2025 00:43:43.151 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/10/2025 00:43:43.157 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/10/2025 00:43:43.572 7f4bd8395ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:43:43.573 7f4bd8395ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:43:43.574 7f4bd8395ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:43:43.574 7f4bd8395ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:43.574 7f4bd8395ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 07/10/2025 00:43:43.574 7f4bd8395ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:43.574 7f4bd8395ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:43:43.574 7f4bd8395ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:43:43.574 7f4bd8395ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:43.575 7f4bd8395ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 07/10/2025 00:43:43.575 7f4bd8395ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:43.575 7f4bd8395ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:43.575 7f4bd8395ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:43:43.575 7f4bd8395ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:43.697 7f4be223a188 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table', created 07/10/2025 00:43:43.701 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/10/2025 00:43:43.706 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/10/2025 00:43:44.633 7f4be223a188 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table' fixed with properties {'gc_grace_seconds': 864000} 07/10/2025 00:43:44.640 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/10/2025 00:43:44.646 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/10/2025 00:43:45.822 7f4be223a188 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table', created 07/10/2025 00:43:45.827 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/10/2025 00:43:45.833 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/10/2025 00:43:46.724 7f4be223a188 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table' fixed with properties {'gc_grace_seconds': 864000} 07/10/2025 00:43:46.729 7f4be223a188 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/10/2025 00:43:46.729 7f4bdf32c240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:46.730 7f4bdf32c240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:43:46.730 7f4bdf32c240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:46.784 7f4be223a188 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.41:5673//', 'pyamqp://guest:guest@10.0.0.61:5673//', 'pyamqp://guest:guest@10.0.0.130:5673//'] 07/10/2025 00:43:46.784 7f4be223a188 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 07/10/2025 00:43:46.784 7f4bd83a98d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:46.785 7f4bd83a98d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Initializing description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 07/10/2025 00:43:46.785 7f4bd83a98d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:46.785 7f4bd83a98d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:46.785 7f4bd83a98d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 07/10/2025 00:43:46.785 7f4bd83a98d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:46.791 7f4be223a188 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 07/10/2025 00:43:46.791 7f4bd83a9ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:46.792 7f4bd83a9ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:43:46.792 7f4bd83a9ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:46.873 7f4be223a188 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 07/10/2025 00:43:46.874 7f4bd839c630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:46.874 7f4bd839c630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 07/10/2025 00:43:46.874 7f4bd839c630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:46.883 7f4be223a188 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 07/10/2025 00:43:46.883 7f4bd839e978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:46.883 7f4bd839e978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:43:46.883 7f4bd839e978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:46.885 7f4bd839c198 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 07/10/2025 00:43:46.885 7f4bd83ca780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:46.885 7f4bd83ca780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 07/10/2025 00:43:46.885 7f4bd83ca780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:46.897 7f4bd839c198 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 07/10/2025 00:43:46.898 7f4bdf3d6048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:46.898 7f4bdf3d6048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:43:46.898 7f4bdf3d6048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:47.574 7f4bd83cbf60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:43:47.575 7f4bd83cbf60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:43:47.575 7f4bd83cbf60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:43:47.575 7f4bd83cbf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:47.575 7f4bd83cbf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/10/2025 00:43:47.575 7f4bd83cbf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:47.575 7f4bd83cbf60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:43:47.576 7f4bd83cbf60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:43:47.576 7f4bd83cbf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:47.576 7f4bd83cbf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:43:47.576 7f4bd83cbf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:47.576 7f4bd83cbf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:47.576 7f4bd83cbf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:43:47.576 7f4bd83cbf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:51.575 7f4bd83cbf60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:43:51.577 7f4bd83cbf60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:43:51.577 7f4bd83cbf60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:43:51.577 7f4bd83cbf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:51.577 7f4bd83cbf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/10/2025 00:43:51.577 7f4bd83cbf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:51.577 7f4bd83cbf60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:43:51.578 7f4bd83cbf60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:43:51.578 7f4bd83cbf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:51.578 7f4bd83cbf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:43:51.578 7f4bd83cbf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:51.578 7f4bd83cbf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:51.578 7f4bd83cbf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:43:51.578 7f4bd83cbf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:55.578 7f4bd83cbf60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:43:55.580 7f4bd83cbf60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:43:55.580 7f4bd83cbf60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:43:55.580 7f4bd83cbf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:55.580 7f4bd83cbf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/10/2025 00:43:55.580 7f4bd83cbf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:55.580 7f4bd83cbf60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:43:55.581 7f4bd83cbf60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:43:55.581 7f4bd83cbf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:55.581 7f4bd83cbf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:43:55.581 7f4bd83cbf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:55.581 7f4bd83cbf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:55.581 7f4bd83cbf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:43:55.581 7f4bd83cbf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:59.581 7f4bd83cbf60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:43:59.582 7f4bd83cbf60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:43:59.582 7f4bd83cbf60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:43:59.582 7f4bd83cbf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:59.582 7f4bd83cbf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/10/2025 00:43:59.582 7f4bd83cbf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:43:59.582 7f4bd83cbf60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:43:59.583 7f4bd83cbf60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:43:59.583 7f4bd83cbf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:59.583 7f4bd83cbf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:43:59.583 7f4bd83cbf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:59.583 7f4bd83cbf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:43:59.583 7f4bd83cbf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:43:59.583 7f4bd83cbf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:44:03.583 7f4bd83cbf60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:44:03.584 7f4bd83cbf60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 07/10/2025 00:44:03.585 7f4bd83cbf60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:44:03.585 7f4bd83cbf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:44:03.585 7f4bd83cbf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/10/2025 00:44:03.585 7f4bd83cbf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:44:03.585 7f4bd83cbf60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 07/10/2025 00:44:03.586 7f4bd83cbf60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 07/10/2025 00:44:03.586 7f4bd83cbf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:44:03.586 7f4bd83cbf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 07/10/2025 00:44:03.586 7f4bd83cbf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:44:03.586 7f4bd83cbf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:44:03.586 7f4bd83cbf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 137 http_port = 8096 start_time = 1752108103502875 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 07/10/2025 00:44:03.586 7f4bd83cbf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 07/10/2025 00:44:06.590 7f4bd839cf60 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3621-3.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.130', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3621-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3621-3.'], 'id_perms': {'created': '2025-07-10T00:44:06.564624', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-07-10T00:44:06.564624', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12177784434006775812, 'uuid_mslong': 12630341299029624390}}, 'parent_type': 'global-system-config', 'parent_uuid': 'cb9e77eb-e213-499d-ae6d-dc900ea07b64', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'af47fc7e-389b-4646-a900-2e6c0c415804'}, 'oper': 'CREATE', 'request-id': 'req-0a21d523-baa5-4c13-a08d-8c5d9f6e6526', 'type': 'analytics_node', 'uuid': 'af47fc7e-389b-4646-a900-2e6c0c415804'} 07/10/2025 00:44:07.255 7f4bd839cf60 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3621-2.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.61', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3621-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3621-2.'], 'id_perms': {'created': '2025-07-10T00:44:07.239690', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-07-10T00:44:07.239690', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13670308365830686748, 'uuid_mslong': 2427555823842643917}}, 'parent_type': 'global-system-config', 'parent_uuid': 'cb9e77eb-e213-499d-ae6d-dc900ea07b64', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '21b06928-f892-4fcd-bdb6-b107c363881c'}, 'oper': 'CREATE', 'request-id': 'req-23dbf2ed-e77d-4cb2-a3fd-90838afbc538', 'type': 'analytics_node', 'uuid': '21b06928-f892-4fcd-bdb6-b107c363881c'} 07/10/2025 00:44:07.350 7f4bd839cf60 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3621-1.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.41', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3621-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3621-1.'], 'id_perms': {'created': '2025-07-10T00:44:07.328592', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-07-10T00:44:07.328592', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11941438302600068393, 'uuid_mslong': 10695066934082226289}}, 'parent_type': 'global-system-config', 'parent_uuid': 'cb9e77eb-e213-499d-ae6d-dc900ea07b64', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '946c82b6-2eff-4871-a5b8-82de25b87d29'}, 'oper': 'CREATE', 'request-id': 'req-70c1e3e3-fab8-44c1-be92-5f2ad52ae7d2', 'type': 'analytics_node', 'uuid': '946c82b6-2eff-4871-a5b8-82de25b87d29'} 07/10/2025 00:44:07.586 7f4bd839c978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 07/10/2025 00:44:07.587 7f4bd839c978 [contrail-device-manager] [INFO]: Session Event: TCP Connected 07/10/2025 00:44:07.588 7f4bd839c978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 07/10/2025 00:44:07.588 7f4bd839c978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:44:07.588 7f4bd839c978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3621-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.41:5673, 10.0.0.61:5673, 10.0.0.130:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.41:9041, 10.0.0.61:9041, 10.0.0.130:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.41:2181, 10.0.0.61:2181, 10.0.0.130:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.41,10.0.0.61,10.0.0.130:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 07/10/2025 00:44:07.588 7f4bd839c978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 07/10/2025 00:44:07.588 7f4bd839c978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect] 07/10/2025 00:44:07.589 7f4bd839c978 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19 07/10/2025 00:44:07.589 7f4bd839c978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 07/10/2025 00:44:07.589 7f4bd839c978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 07/10/2025 00:44:07.589 7f4bd839c978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 07/10/2025 00:44:07.590 7f4bd81cd048 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 07/10/2025 00:44:07.591 7f4bd81cd048 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 07/10/2025 00:44:20.387 7f4bd83b00f0 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001016 07/10/2025 00:44:20.388 7f4bd83b00f0 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 07/10/2025 00:44:20.388 7f4bd83d5978 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 07/10/2025 00:44:20.388 7f4bd83d5978 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 0 07/10/2025 00:44:20.388 7f4bd83d5978 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 07/10/2025 00:44:20.389 7f4bd83d5978 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 07/10/2025 00:44:20.389 7f4bd83d5978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 07/10/2025 00:44:20.389 7f4bd81cbcc0 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 07/10/2025 00:44:20.390 7f4bd81cbcc0 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 07/10/2025 00:44:25.355 7f4bd839cf60 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3621-2.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.61', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3621-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3621-2.'], 'id_perms': {'created': '2025-07-10T00:44:25.342937', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-07-10T00:44:25.342937', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13111413027262333897, 'uuid_mslong': 9746607739359019376}}, 'parent_type': 'global-system-config', 'parent_uuid': 'cb9e77eb-e213-499d-ae6d-dc900ea07b64', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '8742e819-6591-4170-b5f5-18ae596683c9'}, 'oper': 'CREATE', 'request-id': 'req-d690f610-d2cd-44c1-8ee7-e3ed6952b0c2', 'type': 'analytics_alarm_node', 'uuid': '8742e819-6591-4170-b5f5-18ae596683c9'} 07/10/2025 00:44:26.033 7f4bd839cf60 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3621-3.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.130', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3621-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3621-3.'], 'id_perms': {'created': '2025-07-10T00:44:26.014439', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-07-10T00:44:26.014439', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12808646529644161368, 'uuid_mslong': 5018910152836334311}}, 'parent_type': 'global-system-config', 'parent_uuid': 'cb9e77eb-e213-499d-ae6d-dc900ea07b64', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '45a6c031-5369-42e7-b1c1-7427d4fae958'}, 'oper': 'CREATE', 'request-id': 'req-63216db4-3975-447b-9a94-3a13471a0fad', 'type': 'analytics_alarm_node', 'uuid': '45a6c031-5369-42e7-b1c1-7427d4fae958'} 07/10/2025 00:44:26.149 7f4bd839cf60 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3621-1.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.41', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3621-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3621-1.'], 'id_perms': {'created': '2025-07-10T00:44:26.134519', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-07-10T00:44:26.134519', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13476062585546175960, 'uuid_mslong': 13731642250614096935}}, 'parent_type': 'global-system-config', 'parent_uuid': 'cb9e77eb-e213-499d-ae6d-dc900ea07b64', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'be9097df-a299-4427-bb04-9785b66975d8'}, 'oper': 'CREATE', 'request-id': 'req-fbbd1c81-7dbb-4e23-ba38-74221fa72310', 'type': 'analytics_alarm_node', 'uuid': 'be9097df-a299-4427-bb04-9785b66975d8'} 07/10/2025 00:44:31.414 7f4bd83b00f0 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001296 07/10/2025 00:44:31.414 7f4bd83b00f0 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 07/10/2025 00:44:31.415 7f4bd82af048 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 07/10/2025 00:44:31.415 7f4bd82af048 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 2 07/10/2025 00:44:31.415 7f4bd82af048 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 07/10/2025 00:44:31.415 7f4bd82af048 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 07/10/2025 00:44:36.667 7f4bd839cf60 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3621-2.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.61', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3621-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3621-2.'], 'id_perms': {'created': '2025-07-10T00:44:36.651370', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-07-10T00:44:36.651370', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10177376952253257858, 'uuid_mslong': 6649254811097646956}}, 'parent_type': 'global-system-config', 'parent_uuid': 'cb9e77eb-e213-499d-ae6d-dc900ea07b64', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '5c46e605-7c88-476c-8d3d-4e6a804d9882'}, 'oper': 'CREATE', 'request-id': 'req-249baa0e-f920-476c-bb33-13a1dd5b7353', 'type': 'analytics_snmp_node', 'uuid': '5c46e605-7c88-476c-8d3d-4e6a804d9882'} 07/10/2025 00:44:36.861 7f4bd839cf60 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3621-3.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.130', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3621-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3621-3.'], 'id_perms': {'created': '2025-07-10T00:44:36.844645', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-07-10T00:44:36.844645', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12953663449650761045, 'uuid_mslong': 10310607242846620164}}, 'parent_type': 'global-system-config', 'parent_uuid': 'cb9e77eb-e213-499d-ae6d-dc900ea07b64', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '8f16a2a8-d071-4a04-b3c4-a846a8650d55'}, 'oper': 'CREATE', 'request-id': 'req-9f1f0bf2-b84e-4773-968b-278b6e9ef3ea', 'type': 'analytics_snmp_node', 'uuid': '8f16a2a8-d071-4a04-b3c4-a846a8650d55'} 07/10/2025 00:44:37.830 7f4bd839cf60 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3621-1.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.41', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3621-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3621-1.'], 'id_perms': {'created': '2025-07-10T00:44:37.802946', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-07-10T00:44:37.802946', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13524813514387670903, 'uuid_mslong': 2162635858071865755}}, 'parent_type': 'global-system-config', 'parent_uuid': 'cb9e77eb-e213-499d-ae6d-dc900ea07b64', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '1e0339da-d2e5-499b-bbb1-ca3bda7ac777'}, 'oper': 'CREATE', 'request-id': 'req-00b421f9-f2c2-4482-bcba-0b109b0d8d93', 'type': 'analytics_snmp_node', 'uuid': '1e0339da-d2e5-499b-bbb1-ca3bda7ac777'} 07/10/2025 00:46:18.128 7f4bd839cf60 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-9f5cd5c1-dc21-4b0e-b5a7-99f724f9ee5a', 'type': 'global_system_config', 'uuid': 'cb9e77eb-e213-499d-ae6d-dc900ea07b64'} 07/10/2025 00:46:18.171 7f4bd839cf60 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-29250332-8d5a-4545-bebc-480e0a9e10c6', 'type': 'global_system_config', 'uuid': 'cb9e77eb-e213-499d-ae6d-dc900ea07b64'} 07/10/2025 00:46:18.292 7f4bd839cf60 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3621-2'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-3621-2', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3621-2'], 'id_perms': {'created': '2025-07-10T00:46:18.276812', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-07-10T00:46:18.276812', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11334280697037214038, 'uuid_mslong': 15125533433363450363}}, 'parent_type': 'global-system-config', 'parent_uuid': 'cb9e77eb-e213-499d-ae6d-dc900ea07b64', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'd1e8b084-e5ec-49fb-9d4b-743e0b785156', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.18'}, 'oper': 'CREATE', 'request-id': 'req-5b38be09-a5d6-4663-9a45-bb6278c438ff', 'type': 'virtual_router', 'uuid': 'd1e8b084-e5ec-49fb-9d4b-743e0b785156'} 07/10/2025 00:46:18.371 7f4bd839cf60 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3621-1'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-3621-1', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3621-1'], 'id_perms': {'created': '2025-07-10T00:46:18.361222', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-07-10T00:46:18.361222', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9823899781131155529, 'uuid_mslong': 16264869134019808673}}, 'parent_type': 'global-system-config', 'parent_uuid': 'cb9e77eb-e213-499d-ae6d-dc900ea07b64', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'e1b86c4d-6b77-4da1-8855-80cd3b5c3849', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.129'}, 'oper': 'CREATE', 'request-id': 'req-64eddd9e-ae7b-45b7-ba74-37de4a7c79de', 'type': 'virtual_router', 'uuid': 'e1b86c4d-6b77-4da1-8855-80cd3b5c3849'} 07/10/2025 00:46:18.706 7f4bd839cf60 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3621-2', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3621-2', 'vhost0'], 'id_perms': {'created': '2025-07-10T00:46:18.658917', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-07-10T00:46:18.658917', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12900158778927557102, 'uuid_mslong': 9263837041728244698}}, 'parent_type': 'virtual-router', 'parent_uuid': 'd1e8b084-e5ec-49fb-9d4b-743e0b785156', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '808fc2b5-2151-4bda-b306-920fbc1d29ee', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:80:8f:c2:b5:21']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': '30ab93b2-f425-42cb-8584-4f3c7c9ceaf0'}]}, 'oper': 'CREATE', 'request-id': 'req-68046df9-94c6-45cb-94bd-e76ee4e9fca1', 'type': 'virtual_machine_interface', 'uuid': '808fc2b5-2151-4bda-b306-920fbc1d29ee'} 07/10/2025 00:46:18.779 7f4bd839cf60 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3621-2', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-68046df9-94c6-45cb-94bd-e76ee4e9fca1', 'type': 'virtual_machine_interface', 'uuid': '808fc2b5-2151-4bda-b306-920fbc1d29ee'} 07/10/2025 00:46:18.817 7f4bd839cf60 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3621-1', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3621-1', 'vhost0'], 'id_perms': {'created': '2025-07-10T00:46:18.781523', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-07-10T00:46:18.781523', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12465052893684712508, 'uuid_mslong': 9570271604197180517}}, 'parent_type': 'virtual-router', 'parent_uuid': 'e1b86c4d-6b77-4da1-8855-80cd3b5c3849', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '84d06f51-8c2e-4465-acfc-c3906a88443c', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:84:d0:6f:51:8c']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': '30ab93b2-f425-42cb-8584-4f3c7c9ceaf0'}]}, 'oper': 'CREATE', 'request-id': 'req-0534dd9d-e9d1-45c3-8079-e32b1560cf4c', 'type': 'virtual_machine_interface', 'uuid': '84d06f51-8c2e-4465-acfc-c3906a88443c'} 07/10/2025 00:46:18.874 7f4bd839cf60 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3621-1', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-0534dd9d-e9d1-45c3-8079-e32b1560cf4c', 'type': 'virtual_machine_interface', 'uuid': '84d06f51-8c2e-4465-acfc-c3906a88443c'} 07/10/2025 00:46:19.305 7f4bd839cf60 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-df32896a-72a0-4c1e-a3b7-ab59b8304174', 'type': 'access_control_list', 'uuid': 'ad586aca-3859-46fa-981a-c190f7ed8d2e'} 07/10/2025 00:47:09.448 7f4bd83ecd68 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 07/10/2025 00:47:20.353 7f4bd83ecd68 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 07/10/2025 01:45:14.759 7f4bd81cb8d0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map