06/02/2025 02:07:28.866 7fdb02365dc8 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 06/02/2025 02:07:28.866 7fdb02365dc8 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 06/02/2025 02:07:28.866 7fdb02365dc8 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 06/02/2025 02:07:28.868 7fdaf984f588 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 06/02/2025 02:07:28.869 7fdaf984f588 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:07:28.869 7fdaf984f588 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 06/02/2025 02:07:28.869 7fdaf984f588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 06/02/2025 02:07:28.869 7fdaf984f588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:28.870 7fdaf984f588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 06/02/2025 02:07:28.870 7fdaf984f588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:28.870 7fdaf984f588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:28.870 7fdaf984f588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 start_time = 1748830048869418 collector_name = collector_ip = collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 06/02/2025 02:07:28.870 7fdaf984f588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:29.093 7fdb02365dc8 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.129:5673//'] 06/02/2025 02:07:29.169 7fdb02365dc8 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 06/02/2025 02:07:29.173 7fdaf97be630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:29.181 7fdaf97be630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 06/02/2025 02:07:29.181 7fdaf97be630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:29.181 7fdaf97be630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:29.181 7fdaf97be630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 06/02/2025 02:07:29.181 7fdaf97be630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:29.258 7fdb02365dc8 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 06/02/2025 02:07:29.259 7fdaf97d3390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:29.259 7fdaf97d3390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 06/02/2025 02:07:29.259 7fdaf97d3390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:29.285 7fdb02365dc8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 06/02/2025 02:07:29.295 7fdaf97bec18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:29.296 7fdaf97bec18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 06/02/2025 02:07:29.296 7fdaf97bec18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:29.391 7fdb02365dc8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 06/02/2025 02:07:29.394 7fdaf97bec18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:29.395 7fdaf97bec18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 06/02/2025 02:07:29.395 7fdaf97bec18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:29.434 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 06/02/2025 02:07:29.435 7fdb02365dc8 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 06/02/2025 02:07:29.436 7fdb02365dc8 [contrail-device-manager] [WARNING]: initializing cassandra cluster 06/02/2025 02:07:29.437 7fdb02365dc8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 06/02/2025 02:07:29.438 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 06/02/2025 02:07:29.438 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 06/02/2025 02:07:29.439 7fdb0227aac8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 06/02/2025 02:07:29.439 7fdb0227aac8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 06/02/2025 02:07:29.440 7fdaf97d3b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:29.441 7fdaf97d3b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 06/02/2025 02:07:29.441 7fdaf97d3b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:29.442 7fdb02365dc8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 06/02/2025 02:07:29.473 7fdaf97d3d68 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 06/02/2025 02:07:29.473 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 06/02/2025 02:07:29.497 7fdb0227aac8 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 06/02/2025 02:07:30.166 7fdb02365dc8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 06/02/2025 02:07:30.173 7fdaf9751cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:30.173 7fdaf9751cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 06/02/2025 02:07:30.173 7fdaf9751cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:30.409 7fdb02365dc8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 06/02/2025 02:07:30.409 7fdaf9723b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:30.410 7fdaf9723b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 06/02/2025 02:07:30.410 7fdaf9723b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:30.564 7fdb02365dc8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 06/02/2025 02:07:30.893 7fdb02365dc8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 06/02/2025 02:07:32.869 7fdaf9761048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:07:33.268 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/02/2025 02:07:33.274 7fdb02365dc8 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table', created 06/02/2025 02:07:33.287 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/02/2025 02:07:33.904 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/02/2025 02:07:34.111 7fdb02365dc8 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table' fixed with properties {'gc_grace_seconds': 864000} 06/02/2025 02:07:34.115 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/02/2025 02:07:35.779 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/02/2025 02:07:35.794 7fdb02365dc8 [contrail-device-manager] [WARNING]: table 'obj_shared_table', created 06/02/2025 02:07:35.800 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/02/2025 02:07:36.184 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/02/2025 02:07:36.431 7fdb02365dc8 [contrail-device-manager] [WARNING]: table 'obj_shared_table' fixed with properties {'gc_grace_seconds': 864000} 06/02/2025 02:07:36.446 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/02/2025 02:07:36.516 7fdb02365dc8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 06/02/2025 02:07:36.516 7fdb02365dc8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 06/02/2025 02:07:36.517 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 06/02/2025 02:07:36.517 7fdb02365dc8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-juju-os-542-1 06/02/2025 02:07:36.541 7fdaf97d3d68 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 06/02/2025 02:07:36.559 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-juju-os-542-1'] 06/02/2025 02:07:36.681 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 06/02/2025 02:07:36.790 7fdaf97d3d68 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 06/02/2025 02:07:36.797 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-juju-os-542-1'] 06/02/2025 02:07:37.054 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Fetching all containers 06/02/2025 02:07:37.503 7fdb02365dc8 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_api-dnsmasq-1 06/02/2025 02:07:37.766 7fdb02365dc8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-juju-os-542-1.config_queue 06/02/2025 02:07:37.766 7fdb02365dc8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-juju-os-542-1.tftp_queue 06/02/2025 02:07:37.801 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:07:37.809 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 06/02/2025 02:07:37.831 7fdaf97d3d68 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 06/02/2025 02:07:37.872 7fdaf9761048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:07:37.872 7fdaf9761048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:07:37.872 7fdaf9761048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:37.872 7fdaf9761048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 06/02/2025 02:07:37.874 7fdaf9761048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:37.874 7fdaf9761048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:37.874 7fdaf9761048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:07:37.874 7fdaf9761048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:37.874 7fdaf9761048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:37.875 7fdaf9761048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:Zookeeper[] connection down >>, ] >> 06/02/2025 02:07:37.875 7fdaf9761048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:37.875 7fdaf9761048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:37.875 7fdaf9761048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:07:37.875 7fdaf9761048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:37.875 7fdaf9761048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:07:37.875 7fdaf9761048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:07:37.875 7fdaf9761048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:37.875 7fdaf9761048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:07:37.876 7fdaf9761048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:37.876 7fdaf9761048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:37.876 7fdaf9761048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 06/02/2025 02:07:37.876 7fdaf9761048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:37.881 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-juju-os-542-1', 'device_manager_ztp.cn-jenkins-deploy-platform-juju-os-542-1.config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-juju-os-542-1.tftp_queue'] 06/02/2025 02:07:38.268 7fdb02365dc8 [contrail-device-manager] [WARNING]: Process 119 prepared to run in Full mode... 06/02/2025 02:07:38.268 7fdb02365dc8 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 06/02/2025 02:07:38.268 7fdb02365dc8 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 06/02/2025 02:07:38.279 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/local/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page 06/02/2025 02:07:38.279 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 06/02/2025 02:07:38.285 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/local/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page 06/02/2025 02:07:38.285 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 06/02/2025 02:07:38.290 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/local/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 06/02/2025 02:07:38.290 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 06/02/2025 02:07:38.304 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/local/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page 06/02/2025 02:07:38.304 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 06/02/2025 02:07:38.306 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 06/02/2025 02:07:38.307 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 06/02/2025 02:07:38.307 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 06/02/2025 02:07:38.307 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 06/02/2025 02:07:38.307 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 06/02/2025 02:07:38.307 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 06/02/2025 02:07:38.307 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 06/02/2025 02:07:38.307 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 06/02/2025 02:07:38.307 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 06/02/2025 02:07:38.307 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 06/02/2025 02:07:38.307 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 06/02/2025 02:07:38.313 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/local/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 06/02/2025 02:07:38.313 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 06/02/2025 02:07:38.313 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 06/02/2025 02:07:38.313 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 06/02/2025 02:07:38.317 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/local/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 06/02/2025 02:07:38.317 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 06/02/2025 02:07:38.317 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 06/02/2025 02:07:38.318 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/local/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page 06/02/2025 02:07:38.318 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 06/02/2025 02:07:38.318 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 06/02/2025 02:07:38.318 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 06/02/2025 02:07:38.319 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 06/02/2025 02:07:38.319 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 06/02/2025 02:07:38.320 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/local/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page 06/02/2025 02:07:38.320 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 06/02/2025 02:07:38.320 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 06/02/2025 02:07:38.321 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 06/02/2025 02:07:38.321 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 06/02/2025 02:07:38.322 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 06/02/2025 02:07:38.323 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 06/02/2025 02:07:38.323 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 06/02/2025 02:07:38.323 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 06/02/2025 02:07:38.323 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 06/02/2025 02:07:38.323 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 06/02/2025 02:07:38.323 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 06/02/2025 02:07:38.323 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 06/02/2025 02:07:38.323 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 06/02/2025 02:07:38.323 7fdb02365dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 06/02/2025 02:07:38.328 7fdb02365dc8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 06/02/2025 02:07:38.822 7fdaf9508a20 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 06/02/2025 02:07:38.822 7fdaf9508a20 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 06/02/2025 02:07:38.822 7fdaf2a72b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:38.822 7fdaf2a72b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 06/02/2025 02:07:38.823 7fdaf2a72b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:38.832 7fdaf902e2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:38.835 7fdaf902e2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 06/02/2025 02:07:38.835 7fdaf902e2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:38.885 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:07:38.896 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 06/02/2025 02:07:38.920 7fdaf97d3d68 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 06/02/2025 02:07:38.998 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-juju-os-542-1', 'device_manager_ztp.cn-jenkins-deploy-platform-juju-os-542-1.config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-juju-os-542-1.tftp_queue', 'device_manager_ztp.ztp_queue'] 06/02/2025 02:07:40.006 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:07:41.009 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:07:41.994 7fdaf902eac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:41.995 7fdaf902eac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:07:41.996 7fdaf902eac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:41.996 7fdaf902eac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:07:42.010 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:07:43.011 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:07:44.012 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:07:45.013 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:07:46.015 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:07:47.006 7fdaf902eac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:07:47.006 7fdaf902eac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:07:47.006 7fdaf902eac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:47.006 7fdaf902eac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:07:47.006 7fdaf902eac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:47.006 7fdaf902eac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:07:47.007 7fdaf902eac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:07:47.007 7fdaf902eac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:47.007 7fdaf902eac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:07:47.007 7fdaf902eac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:47.007 7fdaf902eac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:47.007 7fdaf902eac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 06/02/2025 02:07:47.007 7fdaf902eac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:47.017 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:07:48.018 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:07:49.020 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:07:50.020 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:07:51.006 7fdaf9140f60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:07:51.022 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:07:52.023 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:07:53.025 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:07:54.027 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:07:55.029 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:07:56.010 7fdaf9140f60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:07:56.010 7fdaf9140f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:07:56.011 7fdaf9140f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:56.011 7fdaf9140f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:07:56.011 7fdaf9140f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:56.011 7fdaf9140f60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:07:56.012 7fdaf9140f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:07:56.012 7fdaf9140f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:56.012 7fdaf9140f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:07:56.012 7fdaf9140f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:56.012 7fdaf9140f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:56.012 7fdaf9140f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 06/02/2025 02:07:56.012 7fdaf9140f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:56.029 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:07:57.031 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:07:58.032 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:07:59.033 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:00.012 7fdaf902eac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:08:00.035 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:01.036 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:02.038 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:03.039 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:04.040 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:05.015 7fdaf902eac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:08:05.015 7fdaf902eac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:08:05.016 7fdaf902eac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:05.016 7fdaf902eac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:08:05.016 7fdaf902eac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:05.016 7fdaf902eac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:08:05.017 7fdaf902eac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:08:05.017 7fdaf902eac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:05.017 7fdaf902eac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:08:05.017 7fdaf902eac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:05.017 7fdaf902eac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:05.017 7fdaf902eac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 06/02/2025 02:08:05.017 7fdaf902eac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:05.042 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:06.042 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:07.044 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:08.045 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:09.015 7fdaf9763cc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:08:09.047 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:09.986 7fdaf914b828 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:08:10.048 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:11.049 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:12.050 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:13.051 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:14.017 7fdaf9763cc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:08:14.017 7fdaf9763cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:08:14.017 7fdaf9763cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:14.018 7fdaf9763cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:08:14.018 7fdaf9763cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:14.018 7fdaf9763cc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:08:14.018 7fdaf9763cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:08:14.018 7fdaf9763cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:14.019 7fdaf9763cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:08:14.019 7fdaf9763cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:14.019 7fdaf9763cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:14.019 7fdaf9763cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 06/02/2025 02:08:14.019 7fdaf9763cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:14.052 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:15.053 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:16.054 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:17.055 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:18.017 7fdaf95f8c18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:08:18.057 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:19.058 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:20.060 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:21.061 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:22.064 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:23.020 7fdaf95f8c18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:08:23.020 7fdaf95f8c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:08:23.020 7fdaf95f8c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:23.020 7fdaf95f8c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:08:23.020 7fdaf95f8c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:23.020 7fdaf95f8c18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:08:23.021 7fdaf95f8c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:08:23.021 7fdaf95f8c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:23.021 7fdaf95f8c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:08:23.022 7fdaf95f8c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:23.022 7fdaf95f8c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:23.022 7fdaf95f8c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 06/02/2025 02:08:23.022 7fdaf95f8c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:23.066 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:24.068 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:25.069 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:25.178 7fdaf82a4eb8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:08:26.070 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:27.019 7fdaf82a4780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:08:27.071 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:28.072 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:29.073 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:30.080 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:31.081 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:32.021 7fdaf82a4780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:08:32.021 7fdaf82a4780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:08:32.021 7fdaf82a4780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:32.021 7fdaf82a4780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:08:32.021 7fdaf82a4780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:32.021 7fdaf82a4780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:08:32.022 7fdaf82a4780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:08:32.022 7fdaf82a4780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:32.022 7fdaf82a4780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:08:32.022 7fdaf82a4780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:32.022 7fdaf82a4780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:32.023 7fdaf82a4780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 06/02/2025 02:08:32.023 7fdaf82a4780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:32.082 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:33.083 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:34.084 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:35.086 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:36.022 7fdaf9147b70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:08:36.087 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:37.088 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:38.089 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:39.090 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:40.091 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:41.024 7fdaf9147b70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:08:41.024 7fdaf9147b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:08:41.024 7fdaf9147b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:41.024 7fdaf9147b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:08:41.024 7fdaf9147b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:41.024 7fdaf9147b70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:08:41.025 7fdaf9147b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:08:41.025 7fdaf9147b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:41.025 7fdaf9147b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:08:41.025 7fdaf9147b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:41.025 7fdaf9147b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:41.025 7fdaf9147b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 06/02/2025 02:08:41.026 7fdaf9147b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:41.092 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:42.093 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:43.095 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:44.097 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:45.026 7fdaff1ef4e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:08:45.097 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:46.098 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:46.993 7fdaf9147b70 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:08:47.099 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:48.100 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:49.102 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:50.031 7fdaff1ef4e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:08:50.031 7fdaff1ef4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:08:50.031 7fdaff1ef4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:50.032 7fdaff1ef4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:08:50.032 7fdaff1ef4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:50.032 7fdaff1ef4e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:08:50.032 7fdaff1ef4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:08:50.032 7fdaff1ef4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:50.032 7fdaff1ef4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:08:50.032 7fdaff1ef4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:50.033 7fdaff1ef4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:50.033 7fdaff1ef4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 06/02/2025 02:08:50.033 7fdaff1ef4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:50.104 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:51.107 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:52.108 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:53.108 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:53.253 7fdb02365dc8 [contrail-device-manager] [WARNING]: NoIdError: default-domain:admin:master-LR 06/02/2025 02:08:53.405 7fdb02365dc8 [contrail-device-manager] [WARNING]: NoIdError: show_interfaces_template 06/02/2025 02:08:53.453 7fdb02365dc8 [contrail-device-manager] [WARNING]: NoIdError: show_config_interfaces_template 06/02/2025 02:08:53.505 7fdb02365dc8 [contrail-device-manager] [WARNING]: NoIdError: show_interfaces_by_names_template 06/02/2025 02:08:53.505 7fdb02365dc8 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 06/02/2025 02:08:53.506 7fdb02365dc8 [contrail-device-manager] [WARNING]: initializing cassandra cluster 06/02/2025 02:08:53.506 7fdb02365dc8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 06/02/2025 02:08:53.506 7fdb03dd4978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:53.507 7fdb03dd4978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 06/02/2025 02:08:53.507 7fdb03dd4978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:53.507 7fdb02365dc8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 06/02/2025 02:08:53.598 7fdb02365dc8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 06/02/2025 02:08:53.599 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:53.600 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Initializing description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 06/02/2025 02:08:53.600 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:53.606 7fdb02365dc8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 06/02/2025 02:08:53.606 7fdaf90cf0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:53.606 7fdaf90cf0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 06/02/2025 02:08:53.606 7fdaf90cf0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:53.630 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/02/2025 02:08:53.692 7fdb02365dc8 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace', created 06/02/2025 02:08:53.695 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/02/2025 02:08:53.701 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/02/2025 02:08:53.758 7fdb02365dc8 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None} 06/02/2025 02:08:53.762 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/02/2025 02:08:53.762 7fdb02365dc8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 06/02/2025 02:08:53.921 7fdb02365dc8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 06/02/2025 02:08:53.929 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/02/2025 02:08:54.031 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:08:54.106 7fdb02365dc8 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table', created 06/02/2025 02:08:54.109 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:54.109 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/02/2025 02:08:54.122 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/02/2025 02:08:54.232 7fdb02365dc8 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table' fixed with properties {'gc_grace_seconds': 864000} 06/02/2025 02:08:54.235 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/02/2025 02:08:54.242 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/02/2025 02:08:54.403 7fdb02365dc8 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table', created 06/02/2025 02:08:54.406 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/02/2025 02:08:54.412 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/02/2025 02:08:54.501 7fdb02365dc8 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table' fixed with properties {'gc_grace_seconds': 864000} 06/02/2025 02:08:54.505 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/02/2025 02:08:54.511 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/02/2025 02:08:54.666 7fdb02365dc8 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table', created 06/02/2025 02:08:54.671 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/02/2025 02:08:54.677 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/02/2025 02:08:54.768 7fdb02365dc8 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table' fixed with properties {'gc_grace_seconds': 864000} 06/02/2025 02:08:54.771 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/02/2025 02:08:54.776 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/02/2025 02:08:54.950 7fdb02365dc8 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table', created 06/02/2025 02:08:54.954 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/02/2025 02:08:54.963 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/02/2025 02:08:55.102 7fdb02365dc8 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table' fixed with properties {'gc_grace_seconds': 864000} 06/02/2025 02:08:55.108 7fdb02365dc8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/02/2025 02:08:55.110 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:55.149 7fdb02365dc8 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.129:5673//'] 06/02/2025 02:08:55.150 7fdb02365dc8 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 06/02/2025 02:08:55.169 7fdb02365dc8 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 06/02/2025 02:08:55.174 7fdb02365dc8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 06/02/2025 02:08:55.189 7fdb02365dc8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 06/02/2025 02:08:55.195 7fdaf919ce10 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 06/02/2025 02:08:55.271 7fdaf919ce10 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 06/02/2025 02:08:56.112 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:57.114 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:57.413 7fdb004b0588 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:08:58.115 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:08:59.033 7fdaf90dd978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:08:59.034 7fdaf90dd978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:08:59.034 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:59.034 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 06/02/2025 02:08:59.034 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:59.034 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:59.034 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:08:59.034 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:59.035 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:59.035 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Initializing description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:RabbitMQ[] connection down >>, ] >> 06/02/2025 02:08:59.035 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:59.035 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:59.035 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:RabbitMQ[] connection down >>, ] >> 06/02/2025 02:08:59.035 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:59.035 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:59.035 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:08:59.035 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:59.035 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:59.035 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:RabbitMQ[] connection down >>, ] >> 06/02/2025 02:08:59.035 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:59.035 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:59.036 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:08:59.036 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:59.036 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:59.036 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:RabbitMQ[] connection down >>, ] >> 06/02/2025 02:08:59.036 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:59.036 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:59.036 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:08:59.036 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:59.036 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:08:59.036 7fdaf90dd978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:08:59.037 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:59.037 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:08:59.037 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:59.037 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:59.037 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:08:59.037 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:59.116 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:00.117 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:01.119 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:02.121 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:03.035 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:09:03.121 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:04.123 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:05.125 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:06.126 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:07.128 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:08.038 7fdaf90dd978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:09:08.038 7fdaf90dd978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:09:08.038 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:08.038 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:09:08.038 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:08.038 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:09:08.039 7fdaf90dd978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:09:08.039 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:08.039 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:09:08.039 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:08.039 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:08.039 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:09:08.039 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:08.129 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:09.130 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:10.132 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:11.133 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:12.039 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:09:12.135 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:13.136 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:14.138 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:15.139 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:16.141 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:17.041 7fdaf90dd978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:09:17.042 7fdaf90dd978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:09:17.042 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:17.042 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:09:17.042 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:17.042 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:09:17.042 7fdaf90dd978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:09:17.043 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:17.043 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:09:17.043 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:17.043 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:17.043 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:09:17.043 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:17.142 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:18.143 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:19.145 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:20.146 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:21.042 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:09:21.147 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:22.149 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:23.150 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:24.152 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:25.153 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:26.045 7fdaf90dd978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:09:26.045 7fdaf90dd978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:09:26.045 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:26.045 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:09:26.045 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:26.046 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:09:26.046 7fdaf90dd978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:09:26.046 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:26.046 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:09:26.047 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:26.047 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:26.047 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:09:26.047 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:26.155 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:27.156 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:28.157 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:29.159 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:30.045 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:09:30.160 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:31.162 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:32.164 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:33.165 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:34.167 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:35.047 7fdaf90dd978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:09:35.048 7fdaf90dd978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:09:35.048 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:35.048 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:09:35.048 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:35.048 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:09:35.049 7fdaf90dd978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:09:35.049 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:35.049 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:09:35.049 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:35.049 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:35.049 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:09:35.049 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:35.167 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:36.169 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:37.170 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:38.172 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:39.049 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:09:39.172 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:40.174 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:41.176 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:42.177 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:43.179 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:44.051 7fdaf90dd978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:09:44.051 7fdaf90dd978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:09:44.051 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:44.052 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:09:44.052 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:44.052 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:09:44.052 7fdaf90dd978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:09:44.053 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:44.053 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:09:44.053 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:44.053 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:44.053 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:09:44.053 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:44.180 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:45.181 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:46.182 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:47.184 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:48.052 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:09:48.185 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:49.187 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:50.189 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:51.190 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:52.192 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:53.056 7fdaf90dd978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:09:53.056 7fdaf90dd978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:09:53.057 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:53.057 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:09:53.057 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:53.057 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:09:53.058 7fdaf90dd978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:09:53.058 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:53.059 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:09:53.059 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:53.059 7fdaf90dd978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:53.059 7fdaf90dd978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:09:53.059 7fdaf90dd978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:53.193 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:54.195 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:55.197 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:56.199 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:57.057 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:09:57.200 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:58.202 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:09:59.203 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:00.205 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:01.206 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:02.059 7fdaf9117eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:10:02.059 7fdaf9117eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:10:02.059 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:02.059 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:10:02.059 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:02.060 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:10:02.060 7fdaf9117eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:10:02.060 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:02.060 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:10:02.060 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:02.060 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:02.060 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:10:02.060 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:02.207 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:03.209 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:04.210 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:05.212 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:06.061 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:10:06.213 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:07.215 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:08.217 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:09.218 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:10.220 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:11.063 7fdaf9117eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:10:11.063 7fdaf9117eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:10:11.064 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:11.064 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:10:11.064 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:11.064 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:10:11.065 7fdaf9117eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:10:11.065 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:11.065 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:10:11.065 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:11.065 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:11.065 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:10:11.065 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:11.222 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:12.223 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:13.225 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:14.227 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:15.064 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:10:15.228 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:16.229 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:17.231 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:18.232 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:18.409 7fdaf95fea20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:10:19.233 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:20.067 7fdaf9117eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:10:20.067 7fdaf9117eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:10:20.068 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:20.068 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:10:20.068 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:20.068 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:10:20.069 7fdaf9117eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:10:20.069 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:20.069 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:10:20.070 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:20.070 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:20.070 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:10:20.070 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:20.235 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:21.236 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:22.237 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:23.239 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:24.068 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:10:24.240 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:25.241 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:26.243 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:27.245 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:28.246 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:29.071 7fdaf9117eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:10:29.071 7fdaf9117eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:10:29.071 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:29.072 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:10:29.072 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:29.072 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:10:29.072 7fdaf9117eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:10:29.072 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:29.072 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:10:29.072 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:29.073 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:29.073 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:10:29.073 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:29.086 7fdaf9117eb8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:10:29.247 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:30.249 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:31.251 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:32.253 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:33.072 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:10:33.254 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:34.255 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:35.257 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:36.258 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:37.259 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:38.075 7fdaf9117eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:10:38.075 7fdaf9117eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:10:38.075 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:38.075 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:10:38.075 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:38.076 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:10:38.076 7fdaf9117eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:10:38.076 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:38.076 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:10:38.076 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:38.076 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:38.076 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:10:38.077 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:38.260 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:39.261 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:40.262 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:41.264 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:42.075 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:10:42.266 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:43.267 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:44.269 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:45.271 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:46.272 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:47.078 7fdaf9117eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:10:47.087 7fdaf9117eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:10:47.087 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:47.087 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:10:47.087 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:47.087 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:10:47.088 7fdaf9117eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:10:47.088 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:47.088 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:10:47.088 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:47.088 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:47.088 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:10:47.088 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:47.273 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:48.274 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:49.275 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:50.277 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:51.079 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:10:51.278 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:52.279 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:53.281 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:54.283 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:55.284 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:56.081 7fdaf9117eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:10:56.081 7fdaf9117eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:10:56.082 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:56.082 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:10:56.082 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:56.082 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:10:56.082 7fdaf9117eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:10:56.082 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:56.082 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:10:56.083 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:56.083 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:56.083 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:10:56.083 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:56.285 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:57.287 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:58.288 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:10:59.290 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:00.082 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:11:00.181 7fdaf82a6f60 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:11:00.230 7fdaf91ad978 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-juju-os-542-1'], 'oper': 'UPDATE', 'request-id': 'req-8d999d32-2f70-4bb8-b4cd-183a385b7552', 'type': 'bgp_router', 'uuid': '118cdf23-919f-48f4-b894-9e396fb2d92c'} 06/02/2025 02:11:00.291 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:01.293 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:02.294 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:03.296 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:04.297 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:05.084 7fdaf9117eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:11:05.084 7fdaf9117eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:11:05.084 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:05.085 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:11:05.085 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:05.085 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:11:05.085 7fdaf9117eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:11:05.085 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:05.085 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:11:05.085 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:05.085 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:05.085 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:11:05.086 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:05.298 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:06.299 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:07.300 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:08.301 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:09.085 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:11:09.303 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:10.304 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:11.305 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:12.307 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:13.308 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:14.087 7fdaf9117eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:11:14.087 7fdaf9117eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:11:14.087 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:14.088 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:11:14.088 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:14.088 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:11:14.088 7fdaf9117eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:11:14.088 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:14.088 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:11:14.088 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:14.088 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:14.089 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:11:14.089 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:14.310 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:15.312 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:16.313 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:17.314 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:18.088 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:11:18.316 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:19.317 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:20.318 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:21.319 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:22.320 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:23.090 7fdaf9117eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:11:23.090 7fdaf9117eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:11:23.090 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:23.090 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:11:23.091 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:23.091 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:11:23.091 7fdaf9117eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:11:23.092 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:23.092 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:11:23.092 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:23.092 7fdaf9117eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:23.092 7fdaf9117eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:11:23.092 7fdaf9117eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:23.321 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:24.322 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:25.324 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:26.326 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:27.091 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:11:27.326 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:28.328 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:29.329 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:30.331 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:31.333 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:32.094 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:11:32.094 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:11:32.094 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:32.094 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:11:32.095 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:32.095 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:11:32.095 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:11:32.095 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:32.095 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:11:32.095 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:32.095 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:32.095 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:11:32.095 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:32.334 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:33.335 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:34.336 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:35.338 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:36.095 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:11:36.340 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:37.341 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:38.342 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:39.343 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:40.344 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:41.097 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:11:41.097 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:11:41.097 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:41.097 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:11:41.098 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:41.098 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:11:41.098 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:11:41.098 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:41.098 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:11:41.098 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:41.098 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:41.099 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:11:41.099 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:41.346 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:42.347 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:43.349 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:44.350 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:45.097 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:11:45.352 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:46.353 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:47.355 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:48.356 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:49.358 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:50.100 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:11:50.101 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:11:50.101 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:50.101 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:11:50.101 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:50.101 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:11:50.102 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:11:50.102 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:50.102 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:11:50.102 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:50.102 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:50.102 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:11:50.102 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:50.359 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:50.543 7fdaf90cf6d8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:11:51.360 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:52.361 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:53.362 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:54.101 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:11:54.363 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:55.365 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:56.366 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:57.367 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:58.369 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:11:59.103 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:11:59.103 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:11:59.103 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:59.104 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:11:59.104 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:59.104 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:11:59.104 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:11:59.104 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:59.104 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:11:59.104 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:59.104 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:59.105 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:11:59.105 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:59.370 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:00.372 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:01.373 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:02.374 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:03.104 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:12:03.375 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:04.376 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:05.377 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:06.379 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:07.381 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:08.106 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:12:08.106 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:12:08.106 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:08.106 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:12:08.106 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:08.106 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:12:08.107 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:12:08.107 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:08.107 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:12:08.107 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:08.107 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:08.107 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:12:08.107 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:08.382 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:09.384 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:10.386 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:11.389 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:12.106 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:12:12.390 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:13.395 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:14.397 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:15.398 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:16.401 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:17.117 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:12:17.118 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:12:17.118 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:17.118 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:12:17.118 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:17.118 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:12:17.119 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:12:17.119 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:17.119 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:12:17.119 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:17.119 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:17.119 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:12:17.119 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:17.403 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:18.408 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:19.409 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:20.411 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:21.118 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:12:21.412 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:22.413 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:23.415 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:24.416 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:25.417 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:26.120 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:12:26.120 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:12:26.120 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:26.121 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:12:26.123 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:26.124 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:12:26.124 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:12:26.124 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:26.125 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:12:26.125 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:26.125 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:26.125 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:12:26.125 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:26.421 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:27.421 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:28.423 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:29.424 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:30.121 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:12:30.426 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:31.427 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:32.429 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:33.429 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:34.431 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:35.123 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:12:35.124 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:12:35.124 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:35.124 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:12:35.124 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:35.124 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:12:35.124 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:12:35.125 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:35.125 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:12:35.125 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:35.125 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:35.125 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:12:35.125 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:35.433 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:36.435 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:37.436 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:38.438 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:39.124 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:12:39.439 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:40.441 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:41.442 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:42.443 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:43.445 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:44.126 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:12:44.127 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:12:44.127 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:44.127 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:12:44.127 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:44.128 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:12:44.128 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:12:44.128 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:44.128 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:12:44.128 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:44.128 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:44.128 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:12:44.128 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:44.446 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:45.447 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:46.449 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:47.450 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:48.127 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:12:48.451 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:49.452 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:50.454 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:51.455 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:52.456 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:53.130 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:12:53.130 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:12:53.130 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:53.130 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:12:53.130 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:53.130 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:12:53.131 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:12:53.131 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:53.131 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:12:53.131 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:53.131 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:53.131 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:12:53.131 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:53.458 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:54.459 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:55.461 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:56.462 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:57.131 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:12:57.463 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:58.464 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:12:59.466 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:00.467 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:01.468 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:02.133 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:13:02.133 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:13:02.133 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:02.134 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:13:02.134 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:02.134 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:13:02.134 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:13:02.134 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:02.134 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:13:02.135 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:02.135 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:02.135 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:13:02.135 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:02.469 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:03.471 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:04.472 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:05.474 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:06.134 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:13:06.476 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:07.477 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:08.479 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:09.480 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:10.482 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:11.136 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:13:11.136 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:13:11.136 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:11.136 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:13:11.137 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:11.137 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:13:11.137 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:13:11.137 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:11.137 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:13:11.138 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:11.138 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:11.138 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:13:11.138 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:11.483 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:12.484 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:13.485 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:14.487 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:15.137 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:13:15.487 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:16.489 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:17.490 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:18.492 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:19.493 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:20.140 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:13:20.140 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:13:20.140 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:20.140 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:13:20.140 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:20.141 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:13:20.141 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:13:20.141 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:20.141 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:13:20.141 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:20.141 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:20.141 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:13:20.141 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:20.495 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:21.497 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:22.498 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:23.500 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:24.141 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:13:24.501 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:25.502 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:26.503 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:27.504 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:28.505 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:29.144 7fdaf91476d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:13:29.144 7fdaf91476d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:13:29.144 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:29.144 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:13:29.144 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:29.144 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:13:29.145 7fdaf91476d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:13:29.145 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:29.145 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:13:29.145 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:29.145 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:29.145 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:13:29.145 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:29.507 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:30.507 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:31.508 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:32.510 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:33.144 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:13:33.512 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:34.513 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:35.514 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:36.515 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:37.517 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:38.147 7fdaf91476d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:13:38.147 7fdaf91476d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:13:38.147 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:38.148 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:13:38.148 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:38.148 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:13:38.148 7fdaf91476d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:13:38.148 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:38.149 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:13:38.149 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:38.149 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:38.149 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:13:38.149 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:38.518 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:39.519 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:40.520 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:41.521 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:42.148 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:13:42.522 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:43.524 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:44.525 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:45.526 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:46.527 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:47.151 7fdaf91476d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:13:47.151 7fdaf91476d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:13:47.151 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:47.151 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:13:47.151 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:47.151 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:13:47.152 7fdaf91476d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:13:47.152 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:47.152 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:13:47.152 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:47.152 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:47.152 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:13:47.152 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:47.529 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:48.530 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:49.531 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:50.532 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:51.151 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:13:51.534 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:52.535 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:53.537 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:54.539 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:55.540 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:56.154 7fdaf91476d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:13:56.154 7fdaf91476d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:13:56.154 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:56.154 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:13:56.154 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:56.154 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:13:56.155 7fdaf91476d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:13:56.155 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:56.155 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:13:56.155 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:56.155 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:56.155 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:13:56.155 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:56.541 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:57.542 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:58.543 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:13:59.545 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:00.155 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:14:00.546 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:01.547 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:02.549 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:03.550 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:04.552 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:05.157 7fdaf91476d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:14:05.157 7fdaf91476d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:14:05.157 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:05.157 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:14:05.157 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:05.158 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:14:05.158 7fdaf91476d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:14:05.158 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:05.158 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:14:05.158 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:05.159 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:05.159 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:14:05.159 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:05.553 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:06.554 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:07.556 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:08.557 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:09.158 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:14:09.558 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:10.560 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:11.561 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:12.562 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:13.564 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:14.161 7fdaf91476d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:14:14.161 7fdaf91476d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:14:14.161 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:14.161 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:14:14.161 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:14.162 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:14:14.162 7fdaf91476d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:14:14.162 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:14.162 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:14:14.162 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:14.163 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:14.163 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:14:14.163 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:14.565 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:15.567 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:16.568 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:17.570 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:18.162 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:14:18.571 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:19.572 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:20.573 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:21.574 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:22.575 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:23.164 7fdaf91476d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:14:23.164 7fdaf91476d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:14:23.164 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:23.164 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:14:23.164 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:23.164 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:14:23.165 7fdaf91476d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:14:23.165 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:23.165 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:14:23.165 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:23.165 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:23.165 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:14:23.165 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:23.576 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:24.578 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:25.580 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:26.581 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:27.164 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:14:27.583 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:28.584 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:29.586 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:30.587 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:31.589 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:32.167 7fdaf91476d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:14:32.167 7fdaf91476d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:14:32.167 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:32.167 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:14:32.168 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:32.168 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:14:32.168 7fdaf91476d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:14:32.168 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:32.168 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:14:32.169 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:32.169 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:32.169 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:14:32.169 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:32.590 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:33.592 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:34.592 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:35.594 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:36.168 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:14:36.595 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:37.596 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:38.598 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:39.599 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:40.601 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:41.170 7fdaf91476d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:14:41.170 7fdaf91476d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:14:41.170 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:41.170 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:14:41.170 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:41.170 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:14:41.171 7fdaf91476d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:14:41.171 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:41.171 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:14:41.171 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:41.171 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:41.171 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:14:41.171 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:41.602 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:42.603 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:43.605 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:44.606 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:45.171 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:14:45.607 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:46.609 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:47.610 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:48.611 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:49.612 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:50.173 7fdaf91476d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:14:50.173 7fdaf91476d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:14:50.173 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:50.173 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:14:50.173 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:50.173 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:14:50.174 7fdaf91476d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:14:50.174 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:50.174 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:14:50.174 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:50.174 7fdaf91476d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:50.174 7fdaf91476d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:14:50.175 7fdaf91476d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:50.614 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:51.616 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:52.617 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:53.618 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:54.174 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:14:54.619 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:55.620 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:56.621 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:57.623 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:58.625 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:14:59.177 7fdafa3966d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:14:59.177 7fdafa3966d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:14:59.177 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:59.177 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:14:59.178 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:59.178 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:14:59.178 7fdafa3966d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:14:59.178 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:59.178 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:14:59.178 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:59.178 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:59.179 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:14:59.179 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:59.627 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:00.628 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:01.630 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:02.632 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:03.178 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:15:03.632 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:04.593 7fdaff1e0eb8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:15:04.633 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:05.634 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:06.636 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:07.637 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:08.180 7fdafa3966d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:15:08.181 7fdafa3966d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:15:08.181 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:08.181 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:15:08.181 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:08.181 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:15:08.182 7fdafa3966d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:15:08.182 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:08.182 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:15:08.182 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:08.182 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:08.182 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:15:08.182 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:08.639 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:09.639 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:10.641 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:11.643 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:12.181 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:15:12.644 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:13.645 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:14.647 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:14.913 7fdaf9600630 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:15:15.648 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:16.650 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:17.183 7fdafa3966d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:15:17.183 7fdafa3966d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:15:17.183 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:17.184 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:15:17.184 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:17.184 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:15:17.184 7fdafa3966d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:15:17.184 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:17.184 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:15:17.184 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:17.184 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:17.185 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:15:17.185 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:17.651 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:18.652 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:19.654 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:20.655 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:21.184 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:15:21.656 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:22.658 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:23.659 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:24.661 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:25.650 7fdb00cc6240 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:15:25.662 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:26.187 7fdafa3966d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:15:26.188 7fdafa3966d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:15:26.188 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:26.188 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:15:26.188 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:26.188 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:15:26.189 7fdafa3966d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:15:26.189 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:26.189 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:15:26.189 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:26.189 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:26.189 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:15:26.189 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:26.663 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:27.665 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:28.667 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:29.668 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:30.188 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:15:30.670 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:31.671 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:32.673 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:33.674 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:34.674 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:35.190 7fdafa3966d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:15:35.190 7fdafa3966d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:15:35.191 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:35.191 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:15:35.191 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:35.191 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:15:35.191 7fdafa3966d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:15:35.191 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:35.191 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:15:35.192 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:35.192 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:35.192 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:15:35.192 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:35.675 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:36.677 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:37.678 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:38.679 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:39.191 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:15:39.680 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:40.681 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:41.682 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:42.683 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:43.684 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:44.193 7fdafa3966d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:15:44.193 7fdafa3966d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:15:44.193 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:44.193 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:15:44.194 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:44.194 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:15:44.194 7fdafa3966d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:15:44.195 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:44.195 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:15:44.195 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:44.195 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:44.195 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:15:44.195 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:44.684 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:45.686 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:46.687 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:47.688 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:48.193 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:15:48.690 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:49.691 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:50.693 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:51.693 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:52.695 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:53.196 7fdafa3966d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:15:53.197 7fdafa3966d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:15:53.197 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:53.197 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:15:53.197 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:53.197 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:15:53.198 7fdafa3966d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:15:53.198 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:53.198 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:15:53.198 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:53.199 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:53.199 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:15:53.199 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:53.697 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:54.698 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:55.699 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:56.701 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:57.197 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:15:57.702 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:58.704 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:15:59.705 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:00.707 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:01.708 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:02.199 7fdafa3966d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:16:02.200 7fdafa3966d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:16:02.200 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:02.200 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:16:02.200 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:02.201 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:16:02.201 7fdafa3966d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:16:02.201 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:02.201 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:16:02.201 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:02.202 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:02.202 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:16:02.202 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:02.710 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:03.711 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:04.713 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:05.714 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:06.200 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:16:06.715 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:07.716 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:08.718 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:09.719 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:10.721 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:11.203 7fdafa3966d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:16:11.203 7fdafa3966d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:16:11.204 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:11.204 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:16:11.204 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:11.204 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:16:11.205 7fdafa3966d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:16:11.205 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:11.205 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:16:11.205 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:11.206 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:11.206 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:16:11.206 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:11.723 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:12.724 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:13.726 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:14.727 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:15.204 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:16:15.728 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:16.730 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:17.732 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:18.734 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:19.735 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:20.206 7fdafa3966d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:16:20.207 7fdafa3966d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:16:20.207 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:20.207 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:16:20.207 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:20.207 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:16:20.207 7fdafa3966d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:16:20.208 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:20.208 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:16:20.208 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:20.208 7fdafa3966d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:20.208 7fdafa3966d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:16:20.208 7fdafa3966d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:20.737 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:21.738 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:22.739 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:23.741 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:24.208 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:16:24.742 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:25.744 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:26.745 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:27.749 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:28.750 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:29.213 7fdaf902e828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:16:29.213 7fdaf902e828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:16:29.214 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:29.214 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:16:29.214 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:29.215 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:16:29.215 7fdaf902e828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:16:29.216 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:29.219 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:16:29.222 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:29.222 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:29.223 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:16:29.223 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:29.752 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:30.756 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:31.759 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:32.765 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:33.214 7fdaf907bf60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:16:33.767 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:34.768 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:35.769 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:36.770 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:37.772 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:38.216 7fdaf907bf60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:16:38.216 7fdaf907bf60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:16:38.216 7fdaf907bf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:38.216 7fdaf907bf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:16:38.216 7fdaf907bf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:38.216 7fdaf907bf60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:16:38.217 7fdaf907bf60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:16:38.217 7fdaf907bf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:38.217 7fdaf907bf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:16:38.217 7fdaf907bf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:38.217 7fdaf907bf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:38.217 7fdaf907bf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:16:38.217 7fdaf907bf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:38.773 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:39.774 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:40.776 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:41.777 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:42.217 7fdaf907bf60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:16:42.778 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:43.779 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:44.783 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:45.785 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:46.786 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:47.220 7fdaf907bf60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:16:47.220 7fdaf907bf60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:16:47.220 7fdaf907bf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:47.220 7fdaf907bf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:16:47.221 7fdaf907bf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:47.221 7fdaf907bf60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:16:47.221 7fdaf907bf60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:16:47.221 7fdaf907bf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:47.221 7fdaf907bf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:16:47.221 7fdaf907bf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:47.221 7fdaf907bf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:47.222 7fdaf907bf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:16:47.222 7fdaf907bf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:47.787 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:48.788 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:49.289 7fdaf907bf60 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:16:49.789 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:50.791 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:51.221 7fdaf907bf60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:16:51.792 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:52.794 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:53.795 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:54.797 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:55.798 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:56.223 7fdaf907bf60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:16:56.223 7fdaf907bf60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:16:56.223 7fdaf907bf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:56.224 7fdaf907bf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:16:56.224 7fdaf907bf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:56.224 7fdaf907bf60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:16:56.225 7fdaf907bf60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:16:56.225 7fdaf907bf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:56.225 7fdaf907bf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:16:56.225 7fdaf907bf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:56.225 7fdaf907bf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:56.225 7fdaf907bf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:16:56.225 7fdaf907bf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:56.799 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:57.801 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:58.802 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:16:59.804 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:00.223 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:17:00.805 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:01.806 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:02.808 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:03.809 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:04.810 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:05.225 7fdafff58780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:17:05.226 7fdafff58780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:17:05.226 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:05.226 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:17:05.226 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:05.226 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:17:05.226 7fdafff58780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:17:05.226 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:05.227 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:17:05.227 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:05.227 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:05.227 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:17:05.227 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:05.812 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:06.814 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:07.815 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:08.817 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:09.226 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:17:09.818 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:10.820 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:11.821 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:12.823 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:13.824 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:14.229 7fdafff58780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:17:14.229 7fdafff58780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:17:14.229 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:14.230 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:17:14.230 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:14.230 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:17:14.230 7fdafff58780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:17:14.230 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:14.230 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:17:14.231 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:14.231 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:14.231 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:17:14.231 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:14.825 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:15.827 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:16.829 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:17.830 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:18.230 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:17:18.831 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:19.832 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:20.834 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:21.835 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:22.837 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:23.231 7fdafff58780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:17:23.232 7fdafff58780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:17:23.232 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:23.232 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:17:23.232 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:23.232 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:17:23.233 7fdafff58780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:17:23.233 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:23.233 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:17:23.233 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:23.233 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:23.233 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:17:23.234 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:23.837 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:24.839 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:25.841 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:26.842 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:27.232 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:17:27.844 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:28.845 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:29.846 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:30.848 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:31.850 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:32.234 7fdafff58780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:17:32.234 7fdafff58780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:17:32.235 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:32.235 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:17:32.235 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:32.235 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:17:32.236 7fdafff58780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:17:32.236 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:32.236 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:17:32.236 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:32.236 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:32.236 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:17:32.237 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:32.851 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:33.853 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:34.855 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:35.857 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:36.235 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:17:36.858 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:37.859 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:38.860 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:39.862 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:40.864 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:41.237 7fdafff58780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:17:41.238 7fdafff58780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:17:41.238 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:41.238 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:17:41.238 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:41.238 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:17:41.239 7fdafff58780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:17:41.239 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:41.239 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:17:41.239 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:41.239 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:41.239 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:17:41.240 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:41.865 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:42.868 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:43.868 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:44.870 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:45.238 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:17:45.872 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:46.873 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:47.874 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:48.877 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:49.879 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:50.240 7fdafff58780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:17:50.241 7fdafff58780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:17:50.241 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:50.241 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:17:50.241 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:50.241 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:17:50.242 7fdafff58780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:17:50.242 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:50.242 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:17:50.242 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:50.242 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:50.242 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:17:50.242 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:50.880 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:51.882 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:52.884 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:53.885 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:54.241 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:17:54.887 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:55.888 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:56.890 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:57.891 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:58.893 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:17:59.243 7fdafff58780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:17:59.243 7fdafff58780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:17:59.243 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:59.244 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:17:59.244 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:59.244 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:17:59.244 7fdafff58780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:17:59.244 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:59.244 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:17:59.244 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:59.244 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:59.245 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:17:59.245 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:59.895 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:00.897 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:01.898 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:02.899 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:03.245 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:18:03.900 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:04.902 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:05.903 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:06.905 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:07.906 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:08.247 7fdafff58780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:18:08.247 7fdafff58780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:18:08.248 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:08.248 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:18:08.248 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:08.248 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:18:08.248 7fdafff58780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:18:08.250 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:08.250 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:18:08.250 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:08.250 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:08.250 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:18:08.251 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:08.907 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:09.909 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:10.910 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:11.911 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:12.248 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:18:12.913 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:13.914 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:14.916 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:15.918 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:16.919 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:17.251 7fdafff58780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:18:17.252 7fdafff58780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:18:17.252 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:17.252 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:18:17.252 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:17.252 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:18:17.253 7fdafff58780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:18:17.253 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:17.253 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:18:17.253 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:17.253 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:17.253 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:18:17.253 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:17.920 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:18.922 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:19.924 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:20.926 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:21.252 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:18:21.927 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:22.929 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:23.930 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:24.932 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:25.933 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:26.256 7fdafff58780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:18:26.256 7fdafff58780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:18:26.256 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:26.256 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:18:26.257 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:26.257 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:18:26.257 7fdafff58780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:18:26.257 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:26.257 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:18:26.257 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:26.257 7fdafff58780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:26.257 7fdafff58780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:18:26.258 7fdafff58780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:26.934 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:27.936 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:28.937 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:29.938 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:30.257 7fdaff0cc828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:18:30.940 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:31.941 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:32.942 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:33.943 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:34.945 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:35.260 7fdaff0cc828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:18:35.260 7fdaff0cc828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:18:35.260 7fdaff0cc828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:35.260 7fdaff0cc828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:18:35.260 7fdaff0cc828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:35.260 7fdaff0cc828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:18:35.261 7fdaff0cc828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:18:35.261 7fdaff0cc828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:35.261 7fdaff0cc828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:18:35.262 7fdaff0cc828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:35.262 7fdaff0cc828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:35.262 7fdaff0cc828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:18:35.262 7fdaff0cc828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:35.947 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:36.949 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:37.951 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:38.952 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:39.260 7fdaff0cc828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:18:39.954 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:40.956 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:41.957 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:42.958 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:43.960 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:44.263 7fdaff0cc828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:18:44.263 7fdaff0cc828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:18:44.263 7fdaff0cc828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:44.264 7fdaff0cc828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:18:44.264 7fdaff0cc828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:44.264 7fdaff0cc828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:18:44.264 7fdaff0cc828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:18:44.264 7fdaff0cc828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:44.265 7fdaff0cc828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:18:44.265 7fdaff0cc828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:44.265 7fdaff0cc828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:44.265 7fdaff0cc828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:18:44.265 7fdaff0cc828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:44.962 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:45.963 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:46.964 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:47.966 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:48.263 7fdaff1d12e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:18:48.967 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:49.968 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:50.970 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:51.972 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:52.973 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:53.266 7fdaff1d12e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:18:53.266 7fdaff1d12e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:18:53.266 7fdaff1d12e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:53.267 7fdaff1d12e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:18:53.267 7fdaff1d12e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:53.267 7fdaff1d12e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:18:53.268 7fdaff1d12e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:18:53.268 7fdaff1d12e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:53.268 7fdaff1d12e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:18:53.268 7fdaff1d12e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:53.268 7fdaff1d12e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:53.268 7fdaff1d12e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:18:53.268 7fdaff1d12e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:53.974 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:54.975 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:55.977 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:56.978 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:57.267 7fdafb966cc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:18:57.979 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:58.980 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:18:59.980 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:00.982 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:01.984 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:02.270 7fdafb966cc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:19:02.270 7fdafb966cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:19:02.270 7fdafb966cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:02.270 7fdafb966cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:19:02.270 7fdafb966cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:02.270 7fdafb966cc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:19:02.271 7fdafb966cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:19:02.271 7fdafb966cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:02.271 7fdafb966cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:19:02.271 7fdafb966cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:02.271 7fdafb966cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:02.271 7fdafb966cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:19:02.271 7fdafb966cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:02.985 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:03.986 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:04.988 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:05.990 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:06.270 7fdafb966cc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:19:06.991 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:07.992 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:08.994 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:09.995 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:10.996 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:11.272 7fdafb966cc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:19:11.273 7fdafb966cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:19:11.273 7fdafb966cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:11.273 7fdafb966cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:19:11.273 7fdafb966cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:11.273 7fdafb966cc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:19:11.273 7fdafb966cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:19:11.274 7fdafb966cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:11.274 7fdafb966cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:19:11.274 7fdafb966cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:11.274 7fdafb966cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:11.274 7fdafb966cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:19:11.274 7fdafb966cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:11.997 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:12.999 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:14.000 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:15.001 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:15.273 7fdafb966cc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:19:16.003 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:17.004 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:18.006 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:19.008 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:20.009 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:20.275 7fdafb966cc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:19:20.275 7fdafb966cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:19:20.275 7fdafb966cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:20.275 7fdafb966cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:19:20.275 7fdafb966cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:20.275 7fdafb966cc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:19:20.276 7fdafb966cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:19:20.276 7fdafb966cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:20.276 7fdafb966cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:19:20.276 7fdafb966cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:20.276 7fdafb966cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:20.276 7fdafb966cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:19:20.277 7fdafb966cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:21.010 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:22.012 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:23.014 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:24.016 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:24.275 7fdaff073438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:19:25.017 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:26.018 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:27.019 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:28.021 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:29.022 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:29.278 7fdaff073438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:19:29.278 7fdaff073438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:19:29.278 7fdaff073438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:29.279 7fdaff073438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:19:29.279 7fdaff073438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:29.279 7fdaff073438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:19:29.280 7fdaff073438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:19:29.280 7fdaff073438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:29.280 7fdaff073438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:19:29.280 7fdaff073438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:29.280 7fdaff073438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:29.281 7fdaff073438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:19:29.281 7fdaff073438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:30.024 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:31.026 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:32.026 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:33.027 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:33.278 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:19:34.029 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:35.031 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:36.031 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:37.033 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:38.034 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:38.281 7fdaf90a8b70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:19:38.281 7fdaf90a8b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:19:38.281 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:38.281 7fdaf90a8b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:19:38.281 7fdaf90a8b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:38.281 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:19:38.282 7fdaf90a8b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:19:38.282 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:38.282 7fdaf90a8b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:19:38.282 7fdaf90a8b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:38.282 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:38.282 7fdaf90a8b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:19:38.282 7fdaf90a8b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:39.036 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:40.036 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:41.038 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:42.040 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:42.281 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:19:43.041 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:44.042 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:45.044 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:46.045 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:47.047 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:47.284 7fdaf90a8b70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:19:47.284 7fdaf90a8b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:19:47.284 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:47.285 7fdaf90a8b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:19:47.285 7fdaf90a8b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:47.285 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:19:47.286 7fdaf90a8b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:19:47.286 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:47.286 7fdaf90a8b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:19:47.286 7fdaf90a8b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:47.286 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:47.286 7fdaf90a8b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:19:47.286 7fdaf90a8b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:48.048 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:49.050 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:50.052 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:51.053 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:51.284 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:19:52.055 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:53.056 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:54.057 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:55.059 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:56.061 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:56.286 7fdaf90a8b70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:19:56.287 7fdaf90a8b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:19:56.287 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:56.287 7fdaf90a8b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:19:56.287 7fdaf90a8b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:56.287 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:19:56.287 7fdaf90a8b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:19:56.288 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:56.288 7fdaf90a8b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:19:56.288 7fdaf90a8b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:56.288 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:56.288 7fdaf90a8b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:19:56.288 7fdaf90a8b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:57.062 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:58.064 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:19:59.065 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:00.066 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:00.288 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:20:01.068 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:02.069 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:03.070 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:04.072 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:05.073 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:05.290 7fdaf90a8b70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:20:05.290 7fdaf90a8b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:20:05.290 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:05.291 7fdaf90a8b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:20:05.291 7fdaf90a8b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:05.291 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:20:05.291 7fdaf90a8b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:20:05.291 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:05.291 7fdaf90a8b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:20:05.291 7fdaf90a8b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:05.291 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:05.291 7fdaf90a8b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:20:05.292 7fdaf90a8b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:06.074 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:07.075 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:08.076 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:09.078 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:09.290 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:20:10.078 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:11.080 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:12.081 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:13.083 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:14.085 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:14.292 7fdaf90a8b70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:20:14.293 7fdaf90a8b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:20:14.293 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:14.293 7fdaf90a8b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:20:14.293 7fdaf90a8b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:14.293 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:20:14.294 7fdaf90a8b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:20:14.294 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:14.294 7fdaf90a8b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:20:14.294 7fdaf90a8b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:14.294 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:14.294 7fdaf90a8b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:20:14.294 7fdaf90a8b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:15.086 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:16.103 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:16.989 7fdaf90a8b70 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:20:17.104 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:18.105 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:18.292 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:20:19.107 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:20.109 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:21.110 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:22.112 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:23.113 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:23.294 7fdaf90a8b70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:20:23.295 7fdaf90a8b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:20:23.295 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:23.295 7fdaf90a8b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:20:23.295 7fdaf90a8b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:23.295 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:20:23.295 7fdaf90a8b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:20:23.296 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:23.296 7fdaf90a8b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:20:23.296 7fdaf90a8b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:23.296 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:23.296 7fdaf90a8b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:20:23.296 7fdaf90a8b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:24.114 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:25.115 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:26.117 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:27.118 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:27.295 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:20:27.708 7fdaf902a438 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:20:28.119 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:29.121 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:30.122 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:31.123 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:32.124 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:32.297 7fdaf90a8b70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:20:32.297 7fdaf90a8b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:20:32.298 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:32.298 7fdaf90a8b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:20:32.298 7fdaf90a8b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:32.298 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:20:32.298 7fdaf90a8b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:20:32.299 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:32.299 7fdaf90a8b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:20:32.299 7fdaf90a8b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:32.299 7fdaf90a8b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:32.299 7fdaf90a8b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:20:32.299 7fdaf90a8b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:33.125 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:34.127 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:35.128 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:36.129 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:36.298 7fdaff0d4240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:20:37.131 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:38.131 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:38.137 7fdaf90e2ac8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:20:39.133 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:40.134 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:41.136 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:41.300 7fdaff0d4240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:20:41.300 7fdaff0d4240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:20:41.300 7fdaff0d4240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:41.300 7fdaff0d4240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:20:41.300 7fdaff0d4240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:41.301 7fdaff0d4240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:20:41.301 7fdaff0d4240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:20:41.301 7fdaff0d4240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:41.301 7fdaff0d4240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:20:41.301 7fdaff0d4240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:41.301 7fdaff0d4240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:41.301 7fdaff0d4240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:20:41.301 7fdaff0d4240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:42.137 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:43.139 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:44.140 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:45.141 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:45.301 7fdaff0d4240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:20:46.142 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:47.144 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:48.145 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:49.146 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:50.148 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:50.304 7fdaff0d4240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:20:50.305 7fdaff0d4240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:20:50.305 7fdaff0d4240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:50.305 7fdaff0d4240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:20:50.305 7fdaff0d4240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:50.306 7fdaff0d4240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:20:50.306 7fdaff0d4240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:20:50.306 7fdaff0d4240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:50.307 7fdaff0d4240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:20:50.307 7fdaff0d4240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:50.307 7fdaff0d4240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:50.307 7fdaff0d4240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:20:50.307 7fdaff0d4240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:51.149 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:52.150 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:53.151 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:54.153 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:54.304 7fdaff0d4240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:20:55.154 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:56.156 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:57.157 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:58.159 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:59.160 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:20:59.306 7fdaff0d4240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:20:59.306 7fdaff0d4240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:20:59.306 7fdaff0d4240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:59.306 7fdaff0d4240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:20:59.306 7fdaff0d4240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:59.306 7fdaff0d4240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:20:59.307 7fdaff0d4240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:20:59.307 7fdaff0d4240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:59.307 7fdaff0d4240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:20:59.307 7fdaff0d4240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:59.307 7fdaff0d4240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:59.307 7fdaff0d4240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:20:59.308 7fdaff0d4240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:00.161 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:01.163 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:02.164 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:03.166 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:03.306 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:21:04.167 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:05.168 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:06.170 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:07.172 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:08.173 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:08.308 7fdaf90e22e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:21:08.309 7fdaf90e22e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:21:08.309 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:08.309 7fdaf90e22e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:21:08.309 7fdaf90e22e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:08.309 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:21:08.310 7fdaf90e22e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:21:08.310 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:08.310 7fdaf90e22e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:21:08.310 7fdaf90e22e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:08.310 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:08.310 7fdaf90e22e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:21:08.311 7fdaf90e22e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:09.174 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:10.175 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:11.177 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:12.178 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:12.310 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:21:13.180 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:14.181 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:15.182 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:16.184 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:17.185 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:17.311 7fdaf90e22e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:21:17.313 7fdaf90e22e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:21:17.313 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:17.313 7fdaf90e22e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:21:17.313 7fdaf90e22e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:17.313 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:21:17.314 7fdaf90e22e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:21:17.314 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:17.314 7fdaf90e22e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:21:17.314 7fdaf90e22e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:17.314 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:17.314 7fdaf90e22e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:21:17.314 7fdaf90e22e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:18.186 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:19.189 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:20.190 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:21.192 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:21.312 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:21:22.193 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:23.201 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:24.202 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:25.203 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:26.204 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:26.314 7fdaf90e22e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:21:26.314 7fdaf90e22e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:21:26.315 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:26.315 7fdaf90e22e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:21:26.316 7fdaf90e22e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:26.316 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:21:26.316 7fdaf90e22e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:21:26.316 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:26.316 7fdaf90e22e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:21:26.317 7fdaf90e22e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:26.317 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:26.317 7fdaf90e22e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:21:26.317 7fdaf90e22e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:27.207 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:28.209 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:29.211 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:30.215 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:30.314 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:21:31.222 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:32.226 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:33.227 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:34.230 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:35.230 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:35.316 7fdaf90e22e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:21:35.317 7fdaf90e22e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:21:35.317 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:35.317 7fdaf90e22e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:21:35.317 7fdaf90e22e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:35.317 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:21:35.318 7fdaf90e22e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:21:35.318 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:35.318 7fdaf90e22e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:21:35.318 7fdaf90e22e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:35.318 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:35.318 7fdaf90e22e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:21:35.318 7fdaf90e22e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:36.231 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:37.233 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:38.234 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:39.235 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:39.317 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:21:40.237 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:41.238 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:42.240 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:43.242 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:44.243 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:44.320 7fdaf90e22e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:21:44.320 7fdaf90e22e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:21:44.320 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:44.320 7fdaf90e22e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:21:44.321 7fdaf90e22e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:44.321 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:21:44.321 7fdaf90e22e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:21:44.321 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:44.322 7fdaf90e22e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:21:44.322 7fdaf90e22e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:44.322 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:44.322 7fdaf90e22e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:21:44.322 7fdaf90e22e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:45.247 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:46.249 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:46.702 7fdaf90e22e8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:21:47.250 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:48.251 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:48.321 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:21:49.252 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:50.253 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:51.255 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:52.257 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:53.258 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:53.323 7fdaf90e22e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:21:53.323 7fdaf90e22e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:21:53.323 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:53.324 7fdaf90e22e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:21:53.324 7fdaf90e22e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:53.324 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:21:53.324 7fdaf90e22e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:21:53.324 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:53.324 7fdaf90e22e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:21:53.325 7fdaf90e22e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:53.325 7fdaf90e22e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:53.325 7fdaf90e22e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:21:53.325 7fdaf90e22e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:54.259 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:55.261 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:56.263 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:57.265 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:57.323 7fdaf90e8828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:21:58.266 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:21:59.267 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:00.268 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:01.270 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:02.271 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:02.326 7fdaf90e8828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:22:02.326 7fdaf90e8828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:22:02.326 7fdaf90e8828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:02.326 7fdaf90e8828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:22:02.327 7fdaf90e8828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:02.327 7fdaf90e8828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:22:02.327 7fdaf90e8828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:22:02.327 7fdaf90e8828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:02.328 7fdaf90e8828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:22:02.328 7fdaf90e8828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:02.328 7fdaf90e8828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:02.328 7fdaf90e8828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:22:02.328 7fdaf90e8828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:03.273 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:04.275 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:05.276 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:06.278 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:06.327 7fdaf90e8828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:22:07.279 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:08.281 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:09.283 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:10.285 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:11.286 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:11.329 7fdaf90e8828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:22:11.329 7fdaf90e8828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:22:11.329 7fdaf90e8828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:11.329 7fdaf90e8828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:22:11.329 7fdaf90e8828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:11.329 7fdaf90e8828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:22:11.330 7fdaf90e8828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:22:11.330 7fdaf90e8828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:11.330 7fdaf90e8828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:22:11.330 7fdaf90e8828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:11.330 7fdaf90e8828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:11.330 7fdaf90e8828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:22:11.330 7fdaf90e8828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:12.287 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:13.289 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:14.290 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:15.291 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:15.329 7fdaf90e8828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:22:16.292 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:17.294 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:18.296 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:19.298 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:20.299 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:20.331 7fdaf90e8828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:22:20.331 7fdaf90e8828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:22:20.331 7fdaf90e8828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:20.331 7fdaf90e8828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:22:20.331 7fdaf90e8828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:20.331 7fdaf90e8828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:22:20.332 7fdaf90e8828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:22:20.332 7fdaf90e8828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:20.332 7fdaf90e8828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:22:20.332 7fdaf90e8828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:20.332 7fdaf90e8828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:20.332 7fdaf90e8828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:22:20.332 7fdaf90e8828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:21.301 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:22.302 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:23.303 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:24.305 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:24.332 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:22:25.307 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:26.308 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:27.309 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:28.309 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:29.311 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:29.334 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:22:29.334 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:22:29.334 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:29.334 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:22:29.334 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:29.334 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:22:29.335 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:22:29.335 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:29.335 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:22:29.335 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:29.335 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:29.335 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:22:29.335 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:30.312 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:31.314 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:32.315 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:33.317 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:33.334 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:22:34.318 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:35.320 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:36.321 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:37.324 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:38.325 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:38.336 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:22:38.336 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:22:38.336 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:38.336 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:22:38.336 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:38.336 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:22:38.337 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:22:38.337 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:38.337 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:22:38.337 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:38.337 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:38.337 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:22:38.337 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:39.327 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:40.329 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:41.330 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:42.332 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:42.336 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:22:43.333 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:44.334 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:45.335 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:46.337 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:47.338 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:47.338 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:22:47.339 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:22:47.339 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:47.339 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:22:47.339 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:47.339 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:22:47.340 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:22:47.340 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:47.340 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:22:47.340 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:47.340 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:47.341 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:22:47.341 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:48.340 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:49.342 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:50.343 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:51.340 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:22:51.344 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:52.347 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:53.348 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:54.350 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:55.351 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:56.343 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:22:56.343 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:22:56.343 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:56.343 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:22:56.343 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:56.343 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:22:56.344 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:22:56.344 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:56.344 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:22:56.344 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:56.344 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:56.344 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:22:56.344 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:56.353 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:57.354 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:58.356 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:22:59.358 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:00.344 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:23:00.359 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:01.361 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:02.362 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:03.364 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:04.365 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:05.346 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:23:05.346 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:23:05.346 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:05.346 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:23:05.346 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:05.346 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:23:05.347 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:23:05.347 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:05.347 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:23:05.347 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:05.347 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:05.347 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:23:05.347 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:05.367 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:06.368 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:07.369 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:08.371 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:09.347 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:23:09.371 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:10.373 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:11.375 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:12.376 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:13.377 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:14.350 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:23:14.350 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:23:14.350 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:14.351 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:23:14.351 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:14.351 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:23:14.351 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:23:14.352 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:14.352 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:23:14.352 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:14.352 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:14.352 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:23:14.352 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:14.378 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:15.379 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:16.381 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:17.383 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:18.351 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:23:18.384 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:19.386 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:20.387 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:21.388 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:22.390 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:23.353 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:23:23.353 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:23:23.353 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:23.353 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:23:23.353 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:23.354 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:23:23.354 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:23:23.354 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:23.354 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:23:23.354 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:23.354 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:23.354 7fdaff1e0eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:23:23.354 7fdaff1e0eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:23.392 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:24.393 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:25.395 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:26.397 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:27.354 7fdb000dd198 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:23:27.398 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:28.400 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:29.401 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:30.402 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:31.404 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:32.357 7fdb000dd198 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:23:32.357 7fdb000dd198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:23:32.357 7fdb000dd198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:32.357 7fdb000dd198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:23:32.357 7fdb000dd198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:32.358 7fdb000dd198 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:23:32.359 7fdb000dd198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:23:32.359 7fdb000dd198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:32.359 7fdb000dd198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:23:32.359 7fdb000dd198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:32.359 7fdb000dd198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:32.360 7fdb000dd198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:23:32.360 7fdb000dd198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:32.405 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:33.406 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:34.409 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:35.410 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:36.358 7fdb000dd198 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:23:36.411 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:37.413 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:38.415 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:39.416 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:40.418 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:41.360 7fdb000dd198 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:23:41.360 7fdb000dd198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:23:41.360 7fdb000dd198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:41.360 7fdb000dd198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:23:41.361 7fdb000dd198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:41.361 7fdb000dd198 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:23:41.361 7fdb000dd198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:23:41.361 7fdb000dd198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:41.361 7fdb000dd198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:23:41.361 7fdb000dd198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:41.361 7fdb000dd198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:41.361 7fdb000dd198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:23:41.361 7fdb000dd198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:41.419 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:42.420 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:43.422 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:44.423 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:45.361 7fdb000dd198 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:23:45.425 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:46.426 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:47.428 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:48.429 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:49.430 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:50.363 7fdb000dd198 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:23:50.364 7fdb000dd198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:23:50.364 7fdb000dd198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:50.364 7fdb000dd198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:23:50.364 7fdb000dd198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:50.364 7fdb000dd198 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:23:50.365 7fdb000dd198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:23:50.365 7fdb000dd198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:50.366 7fdb000dd198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:23:50.366 7fdb000dd198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:50.366 7fdb000dd198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:50.366 7fdb000dd198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:23:50.366 7fdb000dd198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:50.431 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:51.433 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:52.434 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:53.436 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:54.364 7fdb000dd198 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:23:54.437 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:55.440 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:56.441 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:57.443 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:58.445 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:23:59.366 7fdb000dd198 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:23:59.367 7fdb000dd198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:23:59.367 7fdb000dd198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:59.367 7fdb000dd198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:23:59.367 7fdb000dd198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:59.367 7fdb000dd198 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:23:59.368 7fdb000dd198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:23:59.368 7fdb000dd198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:59.368 7fdb000dd198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:23:59.368 7fdb000dd198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:59.368 7fdb000dd198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:59.368 7fdb000dd198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:23:59.368 7fdb000dd198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:59.446 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:00.448 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:01.450 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:02.452 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:03.367 7fdaf936ccc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:24:03.453 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:04.455 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:05.456 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:06.458 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:07.460 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:08.370 7fdaf936ccc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:24:08.371 7fdaf936ccc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:24:08.371 7fdaf936ccc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:08.371 7fdaf936ccc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:24:08.371 7fdaf936ccc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:08.371 7fdaf936ccc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:24:08.372 7fdaf936ccc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:24:08.372 7fdaf936ccc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:08.372 7fdaf936ccc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:24:08.372 7fdaf936ccc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:08.372 7fdaf936ccc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:08.372 7fdaf936ccc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:24:08.372 7fdaf936ccc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:08.462 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:09.464 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:10.466 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:11.467 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:12.371 7fdaf95084e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:24:12.468 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:13.470 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:14.471 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:15.473 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:16.477 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:17.374 7fdaf95084e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:24:17.374 7fdaf95084e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:24:17.374 7fdaf95084e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:17.374 7fdaf95084e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:24:17.374 7fdaf95084e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:17.374 7fdaf95084e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:24:17.375 7fdaf95084e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:24:17.375 7fdaf95084e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:17.375 7fdaf95084e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:24:17.375 7fdaf95084e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:17.375 7fdaf95084e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:17.375 7fdaf95084e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:24:17.375 7fdaf95084e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:17.478 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:18.479 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:19.481 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:20.482 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:21.374 7fdaf95084e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:24:21.483 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:22.485 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:23.487 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:24.489 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:25.491 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:26.376 7fdaf95084e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:24:26.376 7fdaf95084e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:24:26.376 7fdaf95084e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:26.376 7fdaf95084e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:24:26.377 7fdaf95084e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:26.377 7fdaf95084e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:24:26.378 7fdaf95084e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:24:26.378 7fdaf95084e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:26.378 7fdaf95084e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:24:26.378 7fdaf95084e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:26.378 7fdaf95084e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:26.379 7fdaf95084e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:24:26.379 7fdaf95084e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:26.492 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:27.493 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:28.494 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:29.496 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:30.377 7fdaf95084e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:24:30.497 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:31.499 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:32.501 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:33.502 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:34.504 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:35.379 7fdaf95084e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:24:35.380 7fdaf95084e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:24:35.380 7fdaf95084e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:35.380 7fdaf95084e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:24:35.380 7fdaf95084e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:35.380 7fdaf95084e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:24:35.381 7fdaf95084e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:24:35.381 7fdaf95084e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:35.381 7fdaf95084e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:24:35.381 7fdaf95084e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:35.381 7fdaf95084e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:35.381 7fdaf95084e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:24:35.381 7fdaf95084e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:35.506 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:36.506 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:37.508 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:38.509 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:39.380 7fdaf95084e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:24:39.511 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:40.513 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:41.514 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:42.516 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:43.518 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:44.383 7fdaf95084e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:24:44.384 7fdaf95084e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:24:44.384 7fdaf95084e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:44.384 7fdaf95084e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:24:44.384 7fdaf95084e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:44.384 7fdaf95084e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:24:44.384 7fdaf95084e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:24:44.384 7fdaf95084e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:44.385 7fdaf95084e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:24:44.385 7fdaf95084e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:44.385 7fdaf95084e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:44.385 7fdaf95084e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:24:44.385 7fdaf95084e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:44.519 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:45.521 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:46.522 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:47.524 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:48.385 7fdaf95084e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:24:48.524 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:49.526 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:50.528 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:51.529 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:52.531 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:53.387 7fdaf95084e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:24:53.387 7fdaf95084e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:24:53.387 7fdaf95084e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:53.387 7fdaf95084e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:24:53.387 7fdaf95084e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:53.388 7fdaf95084e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:24:53.388 7fdaf95084e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:24:53.389 7fdaf95084e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:53.389 7fdaf95084e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:24:53.389 7fdaf95084e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:53.389 7fdaf95084e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:53.389 7fdaf95084e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:24:53.389 7fdaf95084e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:53.532 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:54.534 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:55.535 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:56.537 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:57.388 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:24:57.539 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:58.541 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:24:59.542 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:00.544 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:01.546 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:02.391 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:25:02.392 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:25:02.393 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:02.393 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:25:02.393 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:02.393 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:25:02.394 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:25:02.394 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:02.394 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:25:02.394 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:02.394 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:02.394 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:25:02.395 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:02.547 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:03.548 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:04.549 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:05.551 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:06.392 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:25:06.552 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:07.554 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:08.555 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:09.556 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:10.558 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:11.395 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:25:11.395 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:25:11.395 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:11.396 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:25:11.396 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:11.396 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:25:11.396 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:25:11.396 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:11.396 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:25:11.397 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:11.397 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:11.397 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:25:11.397 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:11.559 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:12.561 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:13.563 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:14.564 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:15.396 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:25:15.565 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:16.567 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:17.568 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:18.570 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:19.571 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:20.398 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:25:20.399 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:25:20.399 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:20.399 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:25:20.399 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:20.399 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:25:20.399 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:25:20.399 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:20.400 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:25:20.400 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:20.400 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:20.400 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:25:20.400 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:20.573 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:21.574 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:22.576 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:23.577 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:24.399 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:25:24.578 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:25.580 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:26.580 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:27.582 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:28.584 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:29.402 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:25:29.402 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:25:29.402 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:29.402 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:25:29.402 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:29.402 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:25:29.402 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:25:29.403 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:29.403 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:25:29.403 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:29.403 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:29.403 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:25:29.403 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:29.585 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:30.587 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:31.588 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:32.590 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:33.403 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:25:33.591 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:34.593 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:35.594 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:36.597 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:37.599 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:38.405 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:25:38.405 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:25:38.405 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:38.405 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:25:38.405 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:38.405 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:25:38.406 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:25:38.406 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:38.406 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:25:38.406 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:38.406 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:38.407 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:25:38.407 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:38.606 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:39.609 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:40.611 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:41.612 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:42.406 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:25:42.614 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:43.616 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:44.618 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:45.619 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:46.621 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:47.407 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:25:47.407 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:25:47.407 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:47.408 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:25:47.408 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:47.408 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:25:47.408 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:25:47.408 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:47.409 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:25:47.409 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:47.409 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:47.409 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:25:47.409 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:47.622 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:48.623 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:49.624 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:50.626 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:51.408 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:25:51.627 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:52.630 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:53.631 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:54.632 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:55.633 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:56.410 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:25:56.411 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:25:56.411 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:56.411 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:25:56.411 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:56.411 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:25:56.412 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:25:56.412 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:56.412 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:25:56.412 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:56.412 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:56.413 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:25:56.413 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:56.634 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:57.635 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:58.637 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:25:59.650 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:00.411 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:26:00.652 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:01.654 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:02.655 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:03.655 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:04.657 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:05.414 7fdaf90e62e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:26:05.414 7fdaf90e62e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:26:05.415 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:05.415 7fdaf90e62e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:26:05.415 7fdaf90e62e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:05.415 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:26:05.416 7fdaf90e62e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:26:05.416 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:05.416 7fdaf90e62e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:26:05.416 7fdaf90e62e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:05.416 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:05.416 7fdaf90e62e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:26:05.416 7fdaf90e62e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:05.658 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:06.659 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:07.660 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:08.662 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:08.954 7fdaf90e62e8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:26:09.415 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:26:09.662 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:10.664 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:11.665 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:12.667 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:13.668 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:14.418 7fdaf90e62e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:26:14.418 7fdaf90e62e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:26:14.418 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:14.419 7fdaf90e62e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:26:14.419 7fdaf90e62e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:14.419 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:26:14.419 7fdaf90e62e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:26:14.420 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:14.420 7fdaf90e62e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:26:14.420 7fdaf90e62e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:14.420 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:14.420 7fdaf90e62e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:26:14.420 7fdaf90e62e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:14.669 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:15.670 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:16.672 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:17.674 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:18.419 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:26:18.675 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:19.249 7fdaff1d14e0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:26:19.676 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:20.678 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:21.679 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:22.680 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:23.421 7fdaf90e62e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:26:23.421 7fdaf90e62e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:26:23.421 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:23.422 7fdaf90e62e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:26:23.422 7fdaf90e62e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:23.422 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:26:23.422 7fdaf90e62e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:26:23.422 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:23.422 7fdaf90e62e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:26:23.422 7fdaf90e62e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:23.423 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:23.423 7fdaf90e62e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:26:23.423 7fdaf90e62e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:23.682 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:24.683 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:25.685 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:26.686 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:27.422 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:26:27.687 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:28.688 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:29.690 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:29.786 7fdaf90a8048 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:26:30.691 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:31.693 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:32.425 7fdaf90e62e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:26:32.425 7fdaf90e62e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:26:32.425 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:32.425 7fdaf90e62e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:26:32.425 7fdaf90e62e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:32.425 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:26:32.426 7fdaf90e62e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:26:32.426 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:32.426 7fdaf90e62e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:26:32.426 7fdaf90e62e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:32.426 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:32.427 7fdaf90e62e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:26:32.428 7fdaf90e62e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:32.694 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:33.695 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:34.697 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:35.698 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:36.425 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:26:36.699 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:37.700 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:38.702 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:39.703 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:40.705 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:41.427 7fdaf90e62e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:26:41.428 7fdaf90e62e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:26:41.428 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:41.428 7fdaf90e62e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:26:41.429 7fdaf90e62e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:41.429 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:26:41.429 7fdaf90e62e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:26:41.429 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:41.429 7fdaf90e62e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:26:41.429 7fdaf90e62e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:41.430 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:41.430 7fdaf90e62e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:26:41.430 7fdaf90e62e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:41.706 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:42.708 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:43.709 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:44.711 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:45.429 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:26:45.713 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:46.713 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:47.714 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:48.716 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:49.716 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:50.431 7fdaf90e62e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:26:50.432 7fdaf90e62e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:26:50.432 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:50.432 7fdaf90e62e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:26:50.433 7fdaf90e62e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:50.433 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:26:50.434 7fdaf90e62e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:26:50.434 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:50.434 7fdaf90e62e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:26:50.434 7fdaf90e62e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:50.434 7fdaf90e62e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:50.434 7fdaf90e62e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:26:50.434 7fdaf90e62e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:50.718 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:51.719 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:52.720 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:53.722 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:54.432 7fdaf91594e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:26:54.723 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:55.725 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:56.726 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:57.726 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:58.728 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:26:59.434 7fdaf91594e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:26:59.434 7fdaf91594e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:26:59.434 7fdaf91594e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:59.435 7fdaf91594e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:26:59.435 7fdaf91594e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:59.435 7fdaf91594e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:26:59.435 7fdaf91594e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:26:59.435 7fdaf91594e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:59.436 7fdaf91594e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:26:59.436 7fdaf91594e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:59.436 7fdaf91594e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:59.436 7fdaf91594e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:26:59.436 7fdaf91594e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:59.729 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:00.730 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:01.732 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:02.733 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:03.436 7fdaf96bf588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:27:03.735 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:04.736 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:05.737 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:06.738 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:07.740 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:08.438 7fdaf96bf588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:27:08.438 7fdaf96bf588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:27:08.438 7fdaf96bf588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:08.438 7fdaf96bf588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:27:08.438 7fdaf96bf588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:08.439 7fdaf96bf588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:27:08.439 7fdaf96bf588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:27:08.439 7fdaf96bf588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:08.439 7fdaf96bf588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:27:08.440 7fdaf96bf588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:08.440 7fdaf96bf588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:08.441 7fdaf96bf588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:27:08.441 7fdaf96bf588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:08.741 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:09.742 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:10.743 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:11.744 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:12.438 7fdaf96bf588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:27:12.746 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:13.747 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:14.749 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:15.749 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:16.750 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:17.441 7fdaf96bf588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:27:17.441 7fdaf96bf588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:27:17.441 7fdaf96bf588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:17.441 7fdaf96bf588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:27:17.441 7fdaf96bf588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:17.441 7fdaf96bf588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:27:17.442 7fdaf96bf588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:27:17.442 7fdaf96bf588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:17.442 7fdaf96bf588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:27:17.442 7fdaf96bf588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:17.442 7fdaf96bf588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:17.442 7fdaf96bf588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:27:17.442 7fdaf96bf588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:17.752 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:18.753 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:19.755 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:20.757 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:21.442 7fdaf96bf588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:27:21.758 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:22.760 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:23.761 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:24.762 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:25.764 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:26.444 7fdaf96bf588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:27:26.444 7fdaf96bf588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:27:26.444 7fdaf96bf588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:26.444 7fdaf96bf588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:27:26.444 7fdaf96bf588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:26.444 7fdaf96bf588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:27:26.445 7fdaf96bf588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:27:26.445 7fdaf96bf588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:26.445 7fdaf96bf588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:27:26.445 7fdaf96bf588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:26.445 7fdaf96bf588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:26.445 7fdaf96bf588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:27:26.445 7fdaf96bf588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:26.766 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:27.767 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:28.769 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:29.770 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:30.445 7fdaf96bf588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:27:30.771 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:31.772 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:32.774 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:33.775 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:34.075 7fdaf9025240 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:27:34.777 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:35.447 7fdaf96bf588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:27:35.447 7fdaf96bf588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:27:35.447 7fdaf96bf588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:35.447 7fdaf96bf588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:27:35.447 7fdaf96bf588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:35.447 7fdaf96bf588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:27:35.448 7fdaf96bf588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:27:35.448 7fdaf96bf588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:35.448 7fdaf96bf588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:27:35.448 7fdaf96bf588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:35.448 7fdaf96bf588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:35.448 7fdaf96bf588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:27:35.448 7fdaf96bf588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:35.779 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:36.780 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:37.781 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:38.783 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:39.447 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:27:39.783 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:40.784 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:41.785 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:42.787 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:43.788 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:44.450 7fdaf9596390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:27:44.450 7fdaf9596390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:27:44.450 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:44.450 7fdaf9596390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:27:44.450 7fdaf9596390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:44.450 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:27:44.451 7fdaf9596390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:27:44.451 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:44.451 7fdaf9596390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:27:44.451 7fdaf9596390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:44.451 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:44.451 7fdaf9596390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:27:44.452 7fdaf9596390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:44.789 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:45.791 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:46.792 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:47.793 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:48.451 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:27:48.794 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:49.798 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:50.800 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:51.801 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:52.802 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:53.454 7fdaf9596390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:27:53.454 7fdaf9596390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:27:53.454 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:53.454 7fdaf9596390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:27:53.454 7fdaf9596390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:53.454 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:27:53.455 7fdaf9596390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:27:53.455 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:53.455 7fdaf9596390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:27:53.455 7fdaf9596390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:53.455 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:53.456 7fdaf9596390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:27:53.456 7fdaf9596390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:53.804 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:54.805 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:55.808 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:56.810 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:57.455 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:27:57.811 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:58.812 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:27:59.814 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:00.815 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:01.817 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:02.458 7fdaf9596390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:28:02.459 7fdaf9596390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:28:02.459 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:02.459 7fdaf9596390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:28:02.459 7fdaf9596390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:02.459 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:28:02.460 7fdaf9596390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:28:02.460 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:02.460 7fdaf9596390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:28:02.460 7fdaf9596390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:02.460 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:02.460 7fdaf9596390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:28:02.460 7fdaf9596390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:02.819 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:03.820 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:04.823 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:05.825 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:06.459 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:28:06.827 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:07.828 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:08.829 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:09.830 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:10.832 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:11.462 7fdaf9596390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:28:11.462 7fdaf9596390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:28:11.462 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:11.462 7fdaf9596390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:28:11.462 7fdaf9596390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:11.463 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:28:11.463 7fdaf9596390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:28:11.463 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:11.463 7fdaf9596390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:28:11.463 7fdaf9596390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:11.463 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:11.464 7fdaf9596390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:28:11.464 7fdaf9596390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:11.833 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:12.834 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:13.835 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:14.837 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:15.463 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:28:15.838 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:16.839 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:17.840 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:18.842 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:19.843 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:20.466 7fdaf9596390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:28:20.466 7fdaf9596390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:28:20.467 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:20.467 7fdaf9596390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:28:20.467 7fdaf9596390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:20.467 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:28:20.468 7fdaf9596390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:28:20.468 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:20.468 7fdaf9596390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:28:20.468 7fdaf9596390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:20.469 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:20.469 7fdaf9596390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:28:20.469 7fdaf9596390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:20.845 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:21.848 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:22.850 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:23.851 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:24.466 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:28:24.852 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:25.853 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:26.854 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:27.855 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:28.857 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:29.469 7fdaf9596390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:28:29.469 7fdaf9596390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:28:29.469 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:29.469 7fdaf9596390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:28:29.469 7fdaf9596390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:29.469 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:28:29.470 7fdaf9596390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:28:29.470 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:29.470 7fdaf9596390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:28:29.470 7fdaf9596390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:29.470 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:29.470 7fdaf9596390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:28:29.470 7fdaf9596390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:29.858 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:30.859 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:31.860 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:32.861 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:33.469 7fdaf9596630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:28:33.863 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:34.864 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:35.866 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:36.868 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:37.869 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:38.471 7fdaf9596630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:28:38.472 7fdaf9596630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:28:38.472 7fdaf9596630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:38.472 7fdaf9596630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:28:38.472 7fdaf9596630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:38.472 7fdaf9596630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:28:38.473 7fdaf9596630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:28:38.473 7fdaf9596630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:38.474 7fdaf9596630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:28:38.474 7fdaf9596630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:38.474 7fdaf9596630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:38.474 7fdaf9596630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:28:38.474 7fdaf9596630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:38.871 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:39.872 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:40.873 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:41.874 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:42.472 7fdaf9596630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:28:42.875 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:43.876 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:44.878 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:45.879 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:46.881 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:47.475 7fdaf9596630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:28:47.475 7fdaf9596630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:28:47.475 7fdaf9596630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:47.475 7fdaf9596630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:28:47.476 7fdaf9596630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:47.476 7fdaf9596630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:28:47.476 7fdaf9596630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:28:47.476 7fdaf9596630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:47.476 7fdaf9596630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:28:47.476 7fdaf9596630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:47.476 7fdaf9596630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:47.477 7fdaf9596630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:28:47.477 7fdaf9596630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:47.882 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:48.883 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:49.885 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:50.886 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:51.475 7fdaf9596630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:28:51.887 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:52.889 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:53.890 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:54.892 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:55.893 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:56.478 7fdaf9596630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:28:56.478 7fdaf9596630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:28:56.478 7fdaf9596630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:56.478 7fdaf9596630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:28:56.479 7fdaf9596630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:56.479 7fdaf9596630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:28:56.479 7fdaf9596630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:28:56.479 7fdaf9596630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:56.479 7fdaf9596630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:28:56.479 7fdaf9596630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:56.479 7fdaf9596630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:56.479 7fdaf9596630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:28:56.479 7fdaf9596630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:56.894 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:57.896 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:58.897 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:28:59.899 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:00.478 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:29:00.900 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:01.902 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:02.903 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:03.904 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:04.905 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:05.481 7fdaf90a8ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:29:05.481 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:29:05.481 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:05.481 7fdaf90a8ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:29:05.481 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:05.481 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:29:05.482 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:29:05.482 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:05.483 7fdaf90a8ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:29:05.483 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:05.483 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:05.483 7fdaf90a8ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:29:05.483 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:05.907 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:06.908 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:07.909 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:08.911 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:09.481 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:29:09.913 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:10.914 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:11.915 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:12.916 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:13.917 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:14.483 7fdaf90a8ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:29:14.484 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:29:14.484 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:14.484 7fdaf90a8ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:29:14.484 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:14.484 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:29:14.484 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:29:14.484 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:14.485 7fdaf90a8ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:29:14.485 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:14.485 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:14.485 7fdaf90a8ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:29:14.485 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:14.919 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:15.920 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:16.921 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:17.923 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:18.484 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:29:18.923 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:19.925 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:20.926 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:21.928 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:22.929 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:23.486 7fdaf90a8ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:29:23.486 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:29:23.486 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:23.487 7fdaf90a8ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:29:23.487 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:23.487 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:29:23.488 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:29:23.488 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:23.488 7fdaf90a8ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:29:23.488 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:23.488 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:23.488 7fdaf90a8ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:29:23.488 7fdaf90a8ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:23.930 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:24.931 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:25.932 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:26.934 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:27.487 7fdaf9027588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:29:27.935 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:28.937 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:29.938 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:30.940 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:31.941 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:32.489 7fdaf9027588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:29:32.490 7fdaf9027588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:29:32.490 7fdaf9027588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:32.490 7fdaf9027588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:29:32.490 7fdaf9027588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:32.490 7fdaf9027588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:29:32.491 7fdaf9027588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:29:32.491 7fdaf9027588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:32.491 7fdaf9027588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:29:32.491 7fdaf9027588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:32.491 7fdaf9027588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:32.491 7fdaf9027588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:29:32.493 7fdaf9027588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:32.943 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:33.945 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:34.946 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:35.948 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:36.491 7fdaf9027588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:29:36.950 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:37.952 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:38.953 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:39.954 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:40.955 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:41.493 7fdaf9027588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:29:41.493 7fdaf9027588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:29:41.493 7fdaf9027588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:41.494 7fdaf9027588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:29:41.494 7fdaf9027588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:41.494 7fdaf9027588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:29:41.494 7fdaf9027588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:29:41.494 7fdaf9027588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:41.494 7fdaf9027588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:29:41.494 7fdaf9027588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:41.494 7fdaf9027588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:41.494 7fdaf9027588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:29:41.494 7fdaf9027588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:41.956 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:42.957 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:43.958 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:44.960 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:45.494 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:29:45.961 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:46.963 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:47.964 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:48.965 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:49.966 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:50.496 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:29:50.497 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:29:50.497 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:50.497 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:29:50.497 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:50.497 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:29:50.497 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:29:50.497 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:50.498 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:29:50.498 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:50.498 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:50.498 7fdaf90cf6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:29:50.498 7fdaf90cf6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:50.967 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:51.969 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:52.970 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:53.971 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:54.497 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:29:54.972 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:55.974 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:56.975 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:57.976 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:58.978 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:29:59.499 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:29:59.500 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:29:59.500 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:59.500 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:29:59.500 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:59.500 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:29:59.500 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:29:59.501 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:59.501 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:29:59.501 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:59.501 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:59.501 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:29:59.501 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:59.979 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:00.980 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:01.982 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:02.983 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:03.500 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:30:03.984 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:04.986 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:05.987 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:06.988 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:07.989 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:08.502 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:30:08.503 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:30:08.503 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:08.503 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:30:08.503 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:08.504 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:30:08.504 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:30:08.504 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:08.505 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:30:08.505 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:08.505 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:08.506 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:30:08.506 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:08.991 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:09.992 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:10.994 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:11.995 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:12.503 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:30:12.997 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:13.997 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:14.999 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:16.000 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:17.001 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:17.506 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:30:17.506 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:30:17.506 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:17.507 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:30:17.507 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:17.507 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:30:17.507 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:30:17.507 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:17.508 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:30:17.508 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:17.508 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:17.508 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:30:17.508 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:18.002 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:19.004 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:20.006 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:21.007 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:21.507 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:30:22.008 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:23.009 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:24.010 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:25.011 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:26.013 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:26.510 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:30:26.510 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:30:26.510 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:26.510 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:30:26.510 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:26.510 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:30:26.511 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:30:26.511 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:26.511 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:30:26.511 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:26.511 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:26.511 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:30:26.511 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:27.014 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:28.016 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:29.022 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:30.026 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:30.510 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:30:31.031 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:32.044 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:33.049 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:34.050 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:35.052 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:35.513 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:30:35.513 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:30:35.514 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:35.514 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:30:35.514 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:35.514 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:30:35.515 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:30:35.515 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:35.516 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:30:35.518 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:35.518 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:35.518 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:30:35.518 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:36.056 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:37.058 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:38.059 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:39.060 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:39.513 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:30:40.062 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:41.063 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:42.065 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:43.066 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:44.067 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:44.516 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:30:44.517 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:30:44.517 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:44.517 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:30:44.517 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:44.517 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:30:44.518 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:30:44.518 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:44.518 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:30:44.518 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:44.518 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:44.518 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:30:44.518 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:45.069 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:46.070 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:47.071 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:48.073 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:48.518 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:30:49.074 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:50.075 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:51.077 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:52.078 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:52.734 7fdaf90e8588 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:30:53.079 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:53.523 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:30:53.523 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:30:53.523 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:53.523 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:30:53.523 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:53.523 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:30:53.524 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:30:53.524 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:53.524 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:30:53.524 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:53.524 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:53.524 7fdaf2b26eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:30:53.524 7fdaf2b26eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:54.081 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:55.083 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:56.085 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:57.085 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:57.523 7fdaf9600978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:30:58.087 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:30:59.089 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:00.089 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:01.090 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:02.092 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:02.526 7fdaf9600978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:31:02.526 7fdaf9600978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:31:02.526 7fdaf9600978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:02.527 7fdaf9600978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:31:02.527 7fdaf9600978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:02.527 7fdaf9600978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:31:02.527 7fdaf9600978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:31:02.527 7fdaf9600978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:02.527 7fdaf9600978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:31:02.527 7fdaf9600978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:02.527 7fdaf9600978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:02.527 7fdaf9600978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:31:02.528 7fdaf9600978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:03.094 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:03.143 7fdaf9600978 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:31:04.095 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:05.097 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:06.097 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:06.527 7fdaf9600978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:31:07.099 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:08.100 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:09.101 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:10.103 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:11.104 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:11.529 7fdaf9600978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:31:11.529 7fdaf9600978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:31:11.529 7fdaf9600978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:11.530 7fdaf9600978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:31:11.530 7fdaf9600978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:11.530 7fdaf9600978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:31:11.530 7fdaf9600978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:31:11.530 7fdaf9600978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:11.530 7fdaf9600978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:31:11.530 7fdaf9600978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:11.531 7fdaf9600978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:11.531 7fdaf9600978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:31:11.531 7fdaf9600978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:12.105 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:13.107 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:13.709 7fdaf91a64e0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:31:14.108 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:15.109 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:15.530 7fdaf91a64e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:31:16.110 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:17.112 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:18.114 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:19.115 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:20.117 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:20.532 7fdaf91a64e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:31:20.532 7fdaf91a64e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:31:20.532 7fdaf91a64e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:20.533 7fdaf91a64e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:31:20.533 7fdaf91a64e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:20.533 7fdaf91a64e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:31:20.534 7fdaf91a64e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:31:20.534 7fdaf91a64e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:20.534 7fdaf91a64e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:31:20.534 7fdaf91a64e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:20.534 7fdaf91a64e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:20.535 7fdaf91a64e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:31:20.535 7fdaf91a64e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:21.119 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:22.120 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:23.122 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:24.123 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:24.533 7fdaf902e8d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:31:25.124 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:26.126 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:27.127 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:28.128 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:29.130 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:29.534 7fdaf902e8d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:31:29.534 7fdaf902e8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:31:29.535 7fdaf902e8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:29.535 7fdaf902e8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:31:29.535 7fdaf902e8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:29.535 7fdaf902e8d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:31:29.535 7fdaf902e8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:31:29.535 7fdaf902e8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:29.535 7fdaf902e8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:31:29.535 7fdaf902e8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:29.536 7fdaf902e8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:29.536 7fdaf902e8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:31:29.536 7fdaf902e8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:30.132 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:31.132 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:32.134 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:33.135 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:33.535 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:31:34.137 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:35.139 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:36.140 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:37.142 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:38.144 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:38.538 7fdaf902e828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:31:38.538 7fdaf902e828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:31:38.538 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:38.538 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:31:38.538 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:38.538 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:31:38.539 7fdaf902e828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:31:38.539 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:38.539 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:31:38.539 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:38.539 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:38.539 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:31:38.539 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:39.145 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:40.146 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:41.148 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:42.150 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:42.541 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:31:43.151 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:44.153 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:45.153 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:46.154 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:47.156 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:47.544 7fdaf902e828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:31:47.544 7fdaf902e828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:31:47.544 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:47.544 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:31:47.545 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:47.545 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:31:47.545 7fdaf902e828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:31:47.545 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:47.545 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:31:47.546 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:47.546 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:47.546 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:31:47.546 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:48.158 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:49.159 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:50.161 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:51.162 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:51.544 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:31:52.163 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:53.165 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:54.167 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:55.168 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:56.170 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:56.547 7fdaf902e828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:31:56.547 7fdaf902e828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:31:56.547 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:56.547 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:31:56.547 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:56.547 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:31:56.548 7fdaf902e828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:31:56.548 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:56.548 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:31:56.548 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:56.548 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:56.548 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:31:56.548 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:57.171 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:58.172 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:31:59.173 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:00.174 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:00.547 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:32:01.175 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:02.177 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:03.179 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:04.179 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:05.181 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:05.550 7fdaf902e828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:32:05.550 7fdaf902e828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:32:05.550 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:05.550 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:32:05.550 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:05.550 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:32:05.551 7fdaf902e828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:32:05.551 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:05.551 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:32:05.551 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:05.551 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:05.551 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:32:05.551 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:06.182 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:07.184 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:08.185 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:09.186 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:09.551 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:32:10.187 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:11.189 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:12.190 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:13.192 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:14.194 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:14.554 7fdaf902e828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:32:14.554 7fdaf902e828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:32:14.554 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:14.554 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:32:14.554 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:14.554 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:32:14.555 7fdaf902e828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:32:14.555 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:14.555 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:32:14.555 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:14.555 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:14.555 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:32:14.555 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:15.195 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:16.196 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:17.198 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:18.200 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:18.555 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:32:19.201 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:20.202 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:21.203 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:22.205 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:23.205 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:23.557 7fdaf902e828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:32:23.557 7fdaf902e828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:32:23.557 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:23.558 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:32:23.558 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:23.558 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:32:23.558 7fdaf902e828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:32:23.558 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:23.558 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:32:23.559 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:23.559 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:23.559 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:32:23.559 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:24.206 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:25.208 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:26.210 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:27.211 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:27.558 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:32:28.212 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:29.213 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:30.214 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:30.678 7fdaf90e62e8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:32:31.216 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:32.217 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:32.560 7fdaf902e828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:32:32.560 7fdaf902e828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:32:32.560 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:32.560 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:32:32.560 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:32.561 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:32:32.561 7fdaf902e828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:32:32.561 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:32.561 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:32:32.561 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:32.561 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:32.561 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:32:32.561 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:33.219 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:34.220 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:35.221 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:36.223 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:36.561 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:32:37.224 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:38.226 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:39.227 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:40.229 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:41.230 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:41.563 7fdaf902e828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:32:41.564 7fdaf902e828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:32:41.564 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:41.564 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:32:41.564 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:41.564 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:32:41.565 7fdaf902e828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:32:41.565 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:41.565 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:32:41.565 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:41.565 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:41.565 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:32:41.565 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:42.232 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:43.234 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:44.236 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:45.237 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:45.564 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:32:46.238 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:47.239 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:48.241 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:49.242 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:50.244 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:50.567 7fdaf902e828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:32:50.567 7fdaf902e828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:32:50.567 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:50.567 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:32:50.567 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:50.567 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:32:50.568 7fdaf902e828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:32:50.568 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:50.568 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:32:50.568 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:50.568 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:50.568 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:32:50.568 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:51.245 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:52.247 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:53.248 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:54.250 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:54.568 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:32:55.251 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:56.252 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:57.254 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:58.255 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:59.257 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:32:59.569 7fdaf902e828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:32:59.569 7fdaf902e828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:32:59.570 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:59.570 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:32:59.570 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:59.570 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:32:59.570 7fdaf902e828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:32:59.570 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:59.570 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:32:59.571 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:59.571 7fdaf902e828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:59.571 7fdaf902e828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:32:59.571 7fdaf902e828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:00.258 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:01.259 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:02.261 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:03.262 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:03.570 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:33:04.264 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:05.265 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:06.267 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:07.269 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:08.270 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:08.572 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:33:08.573 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:33:08.573 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:08.573 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:33:08.573 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:08.573 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:33:08.574 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:33:08.574 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:08.574 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:33:08.574 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:08.574 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:08.574 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:33:08.574 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:09.272 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:10.274 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:11.276 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:12.277 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:12.573 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:33:13.278 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:14.281 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:15.282 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:16.284 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:17.285 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:17.575 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:33:17.575 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:33:17.575 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:17.575 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:33:17.575 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:17.575 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:33:17.576 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:33:17.576 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:17.576 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:33:17.576 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:17.576 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:17.576 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:33:17.576 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:18.287 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:19.289 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:20.290 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:21.291 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:21.576 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:33:22.292 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:23.294 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:24.296 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:25.297 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:26.299 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:26.578 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:33:26.578 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:33:26.578 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:26.578 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:33:26.578 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:26.578 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:33:26.579 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:33:26.579 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:26.579 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:33:26.579 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:26.579 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:26.579 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:33:26.579 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:27.301 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:28.302 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:29.305 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:30.306 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:30.578 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:33:31.307 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:32.309 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:33.310 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:34.312 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:35.313 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:35.580 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:33:35.580 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:33:35.580 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:35.581 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:33:35.581 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:35.581 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:33:35.581 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:33:35.581 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:35.581 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:33:35.581 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:35.582 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:35.582 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:33:35.582 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:36.315 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:37.317 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:38.318 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:39.319 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:39.580 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:33:40.321 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:41.323 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:42.325 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:43.327 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:44.328 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:44.582 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:33:44.582 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:33:44.583 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:44.583 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:33:44.583 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:44.583 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:33:44.583 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:33:44.583 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:44.583 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:33:44.584 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:44.584 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:44.584 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:33:44.584 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:45.330 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:46.332 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:47.333 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:48.334 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:48.583 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:33:49.336 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:50.338 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:51.340 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:52.341 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:53.343 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:53.586 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:33:53.586 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:33:53.587 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:53.587 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:33:53.587 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:53.587 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:33:53.587 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:33:53.587 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:53.588 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:33:53.588 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:53.588 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:53.588 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:33:53.588 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:54.345 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:55.346 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:56.348 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:57.349 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:57.587 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:33:58.351 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:33:59.352 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:00.353 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:01.355 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:02.356 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:02.590 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:34:02.591 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:34:02.591 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:02.591 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:34:02.591 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:02.591 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:34:02.592 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:34:02.592 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:02.592 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:34:02.592 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:02.592 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:02.592 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:34:02.592 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:03.357 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:04.359 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:05.361 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:06.363 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:06.592 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:34:07.365 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:08.367 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:09.368 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:10.369 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:11.370 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:11.594 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:34:11.594 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:34:11.594 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:11.595 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:34:11.595 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:11.595 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:34:11.595 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:34:11.595 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:11.596 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:34:11.596 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:11.596 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:11.596 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:34:11.596 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:12.372 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:13.374 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:14.375 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:15.377 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:15.595 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:34:16.378 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:17.380 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:18.380 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:19.381 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:20.384 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:20.597 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:34:20.597 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:34:20.597 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:20.597 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:34:20.597 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:20.597 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:34:20.598 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:34:20.598 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:20.598 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:34:20.598 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:20.598 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:20.598 7fdafa3bf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:34:20.598 7fdafa3bf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:21.385 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:22.387 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:23.388 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:24.390 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:24.597 7fdafa3c62e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:34:25.391 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:26.393 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:27.394 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:28.395 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:29.397 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:29.599 7fdafa3c62e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:34:29.599 7fdafa3c62e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:34:29.600 7fdafa3c62e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:29.600 7fdafa3c62e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:34:29.600 7fdafa3c62e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:29.600 7fdafa3c62e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:34:29.601 7fdafa3c62e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:34:29.601 7fdafa3c62e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:29.601 7fdafa3c62e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:34:29.601 7fdafa3c62e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:29.601 7fdafa3c62e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:29.601 7fdafa3c62e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:34:29.601 7fdafa3c62e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:30.399 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:31.401 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:32.401 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:33.402 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:33.601 7fdaf9027240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:34:34.404 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:35.406 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:36.408 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:37.409 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:38.410 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:38.603 7fdaf9027240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:34:38.603 7fdaf9027240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:34:38.603 7fdaf9027240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:38.604 7fdaf9027240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:34:38.604 7fdaf9027240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:38.604 7fdaf9027240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:34:38.605 7fdaf9027240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:34:38.605 7fdaf9027240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:38.605 7fdaf9027240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:34:38.605 7fdaf9027240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:38.605 7fdaf9027240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:38.605 7fdaf9027240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:34:38.605 7fdaf9027240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:39.413 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:40.414 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:41.416 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:42.417 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:42.603 7fdaf9027240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:34:43.418 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:44.419 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:45.421 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:46.422 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:47.425 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:47.606 7fdaf9027240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:34:47.606 7fdaf9027240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:34:47.606 7fdaf9027240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:47.606 7fdaf9027240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:34:47.606 7fdaf9027240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:47.606 7fdaf9027240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:34:47.607 7fdaf9027240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:34:47.607 7fdaf9027240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:47.607 7fdaf9027240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:34:47.607 7fdaf9027240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:47.607 7fdaf9027240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:47.607 7fdaf9027240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:34:47.607 7fdaf9027240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:48.426 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:49.428 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:50.430 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:51.432 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:51.606 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:34:52.433 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:53.434 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:54.435 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:55.437 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:56.438 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:56.608 7fdafcf58630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:34:56.608 7fdafcf58630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:34:56.608 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:56.609 7fdafcf58630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:34:56.609 7fdafcf58630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:56.609 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:34:56.609 7fdafcf58630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:34:56.609 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:56.610 7fdafcf58630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:34:56.610 7fdafcf58630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:56.610 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:56.610 7fdafcf58630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:34:56.610 7fdafcf58630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:57.441 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:58.443 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:34:59.444 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:00.446 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:00.609 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:35:01.448 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:02.449 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:03.451 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:04.452 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:05.453 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:05.611 7fdafcf58630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:35:05.611 7fdafcf58630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:35:05.611 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:05.612 7fdafcf58630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:35:05.612 7fdafcf58630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:05.612 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:35:05.612 7fdafcf58630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:35:05.612 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:05.612 7fdafcf58630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:35:05.613 7fdafcf58630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:05.613 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:05.613 7fdafcf58630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:35:05.613 7fdafcf58630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:06.455 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:07.457 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:08.458 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:09.459 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:09.612 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:35:10.461 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:11.462 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:12.465 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:13.467 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:14.468 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:14.615 7fdafcf58630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:35:14.615 7fdafcf58630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:35:14.615 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:14.615 7fdafcf58630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:35:14.615 7fdafcf58630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:14.615 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:35:14.616 7fdafcf58630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:35:14.616 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:14.616 7fdafcf58630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:35:14.616 7fdafcf58630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:14.616 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:14.616 7fdafcf58630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:35:14.617 7fdafcf58630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:15.470 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:16.471 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:17.474 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:18.475 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:18.616 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:35:19.477 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:20.479 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:21.480 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:22.482 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:23.484 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:23.617 7fdafcf58630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:35:23.618 7fdafcf58630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:35:23.618 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:23.618 7fdafcf58630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:35:23.618 7fdafcf58630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:23.618 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:35:23.619 7fdafcf58630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:35:23.619 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:23.619 7fdafcf58630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:35:23.619 7fdafcf58630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:23.619 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:23.619 7fdafcf58630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:35:23.619 7fdafcf58630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:24.486 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:25.487 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:26.489 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:27.491 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:27.618 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:35:28.493 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:29.494 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:30.495 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:31.496 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:32.497 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:32.620 7fdafcf58630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:35:32.620 7fdafcf58630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:35:32.621 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:32.621 7fdafcf58630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:35:32.621 7fdafcf58630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:32.621 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:35:32.621 7fdafcf58630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:35:32.621 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:32.622 7fdafcf58630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:35:32.622 7fdafcf58630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:32.622 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:32.622 7fdafcf58630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:35:32.622 7fdafcf58630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:33.499 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:34.501 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:35.502 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:36.504 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:36.620 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:35:37.505 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:38.507 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:39.508 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:40.509 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:41.511 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:41.623 7fdafcf58630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:35:41.623 7fdafcf58630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:35:41.623 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:41.623 7fdafcf58630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:35:41.623 7fdafcf58630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:41.623 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:35:41.624 7fdafcf58630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:35:41.624 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:41.624 7fdafcf58630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:35:41.624 7fdafcf58630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:41.624 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:41.624 7fdafcf58630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:35:41.624 7fdafcf58630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:42.513 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:43.514 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:44.516 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:45.517 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:45.623 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:35:46.518 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:47.520 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:48.521 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:49.523 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:50.524 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:50.625 7fdafcf58630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:35:50.625 7fdafcf58630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:35:50.626 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:50.626 7fdafcf58630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:35:50.626 7fdafcf58630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:50.626 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:35:50.627 7fdafcf58630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:35:50.627 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:50.627 7fdafcf58630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:35:50.627 7fdafcf58630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:50.627 7fdafcf58630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:50.627 7fdafcf58630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:35:50.627 7fdafcf58630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:51.526 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:52.528 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:53.530 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:54.532 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:54.626 7fdaf91a6978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:35:55.534 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:56.535 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:57.537 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:58.539 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:59.541 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:35:59.628 7fdaf91a6978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:35:59.628 7fdaf91a6978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:35:59.628 7fdaf91a6978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:59.628 7fdaf91a6978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:35:59.628 7fdaf91a6978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:59.628 7fdaf91a6978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:35:59.629 7fdaf91a6978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:35:59.629 7fdaf91a6978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:59.629 7fdaf91a6978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:35:59.629 7fdaf91a6978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:59.629 7fdaf91a6978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:59.629 7fdaf91a6978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:35:59.629 7fdaf91a6978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:00.542 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:01.544 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:02.547 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:03.549 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:03.628 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:36:04.550 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:05.552 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:06.554 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:07.557 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:08.558 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:08.630 7fdaf9596390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:36:08.630 7fdaf9596390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:36:08.630 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:08.630 7fdaf9596390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:36:08.630 7fdaf9596390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:08.631 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:36:08.631 7fdaf9596390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:36:08.631 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:08.631 7fdaf9596390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:36:08.632 7fdaf9596390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:08.632 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:08.632 7fdaf9596390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:36:08.632 7fdaf9596390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:09.010 7fdaf9596390 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:36:09.559 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:10.561 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:11.562 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:12.564 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:12.631 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:36:13.566 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:14.567 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:15.569 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:16.570 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:17.571 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:17.633 7fdaf9596390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:36:17.633 7fdaf9596390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:36:17.634 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:17.634 7fdaf9596390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:36:17.634 7fdaf9596390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:17.634 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:36:17.634 7fdaf9596390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:36:17.634 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:17.634 7fdaf9596390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:36:17.635 7fdaf9596390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:17.635 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:17.635 7fdaf9596390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:36:17.635 7fdaf9596390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:18.574 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:19.449 7fdaf9596390 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:36:19.575 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:20.576 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:21.579 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:21.634 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:36:22.581 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:23.582 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:24.585 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:25.588 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:26.589 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:26.637 7fdaf9596390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:36:26.637 7fdaf9596390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:36:26.637 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:26.637 7fdaf9596390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:36:26.638 7fdaf9596390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:26.638 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:36:26.638 7fdaf9596390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:36:26.638 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:26.639 7fdaf9596390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:36:26.639 7fdaf9596390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:26.639 7fdaf9596390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:26.642 7fdaf9596390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:36:26.642 7fdaf9596390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:27.637 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:28.639 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:29.640 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:30.637 7fdaf919cc18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:36:30.641 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:31.643 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:32.646 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:33.647 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:34.649 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:34.974 7fdaf9117eb8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:36:35.640 7fdaf919cc18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:36:35.640 7fdaf919cc18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:36:35.640 7fdaf919cc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:35.640 7fdaf919cc18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:36:35.641 7fdaf919cc18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:35.641 7fdaf919cc18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:36:35.641 7fdaf919cc18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:36:35.641 7fdaf919cc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:35.641 7fdaf919cc18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:36:35.641 7fdaf919cc18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:35.641 7fdaf919cc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:35.641 7fdaf919cc18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:36:35.642 7fdaf919cc18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:35.650 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:36.651 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:37.653 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:38.655 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:39.642 7fdaf958dac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:36:39.656 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:40.658 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:41.659 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:42.661 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:43.662 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:44.645 7fdaf958dac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:36:44.645 7fdaf958dac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:36:44.645 7fdaf958dac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:44.645 7fdaf958dac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:36:44.645 7fdaf958dac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:44.645 7fdaf958dac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:36:44.646 7fdaf958dac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:36:44.646 7fdaf958dac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:44.647 7fdaf958dac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:36:44.647 7fdaf958dac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:44.647 7fdaf958dac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:44.647 7fdaf958dac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:36:44.647 7fdaf958dac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:44.664 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:45.665 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:46.667 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:47.669 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:48.646 7fdaf958dac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:36:48.670 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:49.672 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:50.673 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:51.674 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:52.676 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:53.648 7fdaf958dac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:36:53.649 7fdaf958dac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:36:53.649 7fdaf958dac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:53.649 7fdaf958dac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:36:53.649 7fdaf958dac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:53.649 7fdaf958dac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:36:53.650 7fdaf958dac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:36:53.650 7fdaf958dac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:53.650 7fdaf958dac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:36:53.650 7fdaf958dac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:53.650 7fdaf958dac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:53.650 7fdaf958dac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:36:53.650 7fdaf958dac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:53.676 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:54.678 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:55.679 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:56.681 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:57.649 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:36:57.682 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:58.683 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:36:59.685 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:00.686 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:01.688 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:02.651 7fdaf2b21978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:37:02.651 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:37:02.652 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:02.652 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:37:02.652 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:02.652 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:37:02.653 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:37:02.653 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:02.653 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:37:02.653 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:02.653 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:02.654 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:37:02.654 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:02.689 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:03.690 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:04.692 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:05.694 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:06.651 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:37:06.695 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:07.697 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:08.698 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:09.700 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:10.701 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:11.653 7fdaf2b21978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:37:11.654 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:37:11.654 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:11.654 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:37:11.654 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:11.654 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:37:11.654 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:37:11.655 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:11.655 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:37:11.655 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:11.655 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:11.655 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:37:11.655 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:11.702 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:12.704 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:13.706 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:14.707 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:15.655 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:37:15.708 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:16.710 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:17.712 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:18.714 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:19.715 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:20.659 7fdaf2b21978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:37:20.659 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:37:20.660 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:20.660 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:37:20.660 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:20.660 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:37:20.661 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:37:20.661 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:20.661 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:37:20.661 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:20.661 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:20.661 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:37:20.661 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:20.717 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:21.718 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:22.720 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:23.722 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:24.660 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:37:24.723 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:25.725 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:26.727 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:27.728 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:28.730 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:29.663 7fdaf2b21978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:37:29.663 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:37:29.663 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:29.663 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:37:29.663 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:29.663 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:37:29.664 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:37:29.664 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:29.664 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:37:29.664 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:29.664 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:29.665 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:37:29.665 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:29.731 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:30.732 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:31.734 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:32.734 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:33.663 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:37:33.736 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:34.738 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:35.740 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:36.741 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:37.742 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:38.667 7fdaf2b21978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:37:38.667 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:37:38.667 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:38.667 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:37:38.667 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:38.667 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:37:38.668 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:37:38.668 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:38.668 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:37:38.668 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:38.668 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:38.668 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:37:38.668 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:38.743 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:39.744 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:40.746 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:41.747 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:42.668 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:37:42.748 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:43.750 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:44.751 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:45.753 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:46.754 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:47.671 7fdaf2b21978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:37:47.671 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:37:47.671 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:47.671 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:37:47.672 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:47.672 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:37:47.672 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:37:47.672 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:47.672 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:37:47.673 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:47.673 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:47.673 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:37:47.673 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:47.755 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:48.757 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:49.758 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:50.759 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:51.673 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:37:51.760 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:52.762 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:53.764 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:54.765 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:55.767 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:56.675 7fdaf2b21978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:37:56.676 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:37:56.676 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:56.676 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:37:56.676 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:56.676 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:37:56.677 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:37:56.677 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:56.677 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:37:56.677 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:56.677 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:56.677 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:37:56.677 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:56.768 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:57.770 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:58.772 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:37:59.773 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:00.676 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:38:00.774 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:01.776 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:02.777 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:03.779 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:04.782 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:05.680 7fdaf2b21978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:38:05.680 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:38:05.680 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:05.680 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:38:05.680 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:05.681 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:38:05.681 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:38:05.681 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:05.681 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:38:05.681 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:05.681 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:05.681 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:38:05.681 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:05.784 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:06.785 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:07.787 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:08.790 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:09.681 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:38:09.791 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:10.793 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:11.794 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:12.796 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:13.797 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:14.684 7fdaf2b21978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:38:14.684 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:38:14.684 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:14.684 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:38:14.684 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:14.684 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:38:14.685 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:38:14.685 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:14.685 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:38:14.685 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:14.685 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:14.685 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:38:14.685 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:14.798 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:15.800 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:16.801 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:17.803 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:18.685 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:38:18.805 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:19.072 7fdaf972a4e0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:38:19.806 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:20.807 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:21.809 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:22.811 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:23.687 7fdaf2b21978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:38:23.687 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:38:23.687 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:23.687 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:38:23.688 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:23.688 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:38:23.688 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:38:23.688 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:23.688 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:38:23.688 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:23.688 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:23.688 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:38:23.688 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:23.813 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:24.814 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:25.817 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:26.818 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:27.688 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:38:27.819 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:28.821 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:29.822 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:30.824 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:31.826 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:32.691 7fdaf91a6a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:38:32.691 7fdaf91a6a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:38:32.691 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:32.692 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:38:32.692 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:32.692 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:38:32.692 7fdaf91a6a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:38:32.692 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:32.693 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:38:32.693 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:32.693 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:32.693 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:38:32.693 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:32.826 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:33.828 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:34.830 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:35.832 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:36.692 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:38:36.833 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:37.835 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:38.837 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:39.838 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:40.841 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:41.694 7fdaf91a6a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:38:41.694 7fdaf91a6a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:38:41.694 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:41.694 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:38:41.695 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:41.695 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:38:41.695 7fdaf91a6a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:38:41.695 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:41.695 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:38:41.695 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:41.695 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:41.695 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:38:41.696 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:41.842 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:42.843 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:43.845 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:44.846 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:45.695 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:38:45.847 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:46.849 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:47.850 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:48.853 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:49.854 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:50.698 7fdaf91a6a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:38:50.698 7fdaf91a6a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:38:50.698 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:50.698 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:38:50.699 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:50.699 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:38:50.699 7fdaf91a6a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:38:50.699 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:50.700 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:38:50.700 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:50.700 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:50.700 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:38:50.700 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:50.855 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:51.857 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:52.859 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:53.860 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:54.699 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:38:54.861 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:55.863 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:56.864 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:57.865 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:58.867 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:38:59.703 7fdaf91a6a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:38:59.703 7fdaf91a6a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:38:59.703 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:59.703 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:38:59.703 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:59.703 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:38:59.704 7fdaf91a6a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:38:59.704 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:59.704 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:38:59.704 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:59.704 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:59.704 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:38:59.704 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:59.868 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:00.869 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:01.872 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:02.873 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:03.703 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:39:03.875 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:04.876 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:05.878 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:06.879 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:07.880 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:08.706 7fdaf91a6a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:39:08.706 7fdaf91a6a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:39:08.706 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:08.706 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:39:08.706 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:08.707 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:39:08.707 7fdaf91a6a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:39:08.707 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:08.707 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:39:08.707 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:08.707 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:08.707 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:39:08.708 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:08.882 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:09.884 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:10.886 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:11.887 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:12.707 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:39:12.889 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:13.891 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:14.893 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:15.894 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:16.896 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:17.710 7fdaf91a6a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:39:17.710 7fdaf91a6a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:39:17.710 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:17.710 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:39:17.710 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:17.710 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:39:17.711 7fdaf91a6a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:39:17.711 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:17.711 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:39:17.711 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:17.711 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:17.712 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:39:17.712 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:17.898 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:18.900 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:19.901 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:20.903 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:21.711 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:39:21.905 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:22.906 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:23.907 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:24.909 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:25.911 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:26.713 7fdaf91a6a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:39:26.713 7fdaf91a6a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:39:26.713 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:26.713 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:39:26.714 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:26.714 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:39:26.714 7fdaf91a6a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:39:26.714 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:26.715 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:39:26.715 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:26.715 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:26.715 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:39:26.715 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:26.913 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:27.914 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:28.916 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:29.917 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:30.713 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:39:30.918 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:31.920 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:32.921 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:33.921 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:34.923 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:35.715 7fdaf91a6a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:39:35.716 7fdaf91a6a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:39:35.716 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:35.716 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:39:35.716 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:35.716 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:39:35.717 7fdaf91a6a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:39:35.717 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:35.717 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:39:35.717 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:35.717 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:35.717 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:39:35.717 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:35.924 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:36.925 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:37.927 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:38.929 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:39.716 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:39:39.930 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:40.932 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:41.933 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:42.934 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:43.935 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:44.719 7fdaf91a6a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:39:44.719 7fdaf91a6a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:39:44.719 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:44.719 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:39:44.719 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:44.719 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:39:44.720 7fdaf91a6a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:39:44.720 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:44.720 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:39:44.720 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:44.720 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:44.721 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:39:44.721 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:44.936 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:45.937 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:46.939 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:47.941 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:48.719 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:39:48.942 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:49.944 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:50.945 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:51.946 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:52.948 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:53.721 7fdaf91a6a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:39:53.721 7fdaf91a6a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:39:53.721 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:53.721 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:39:53.722 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:53.722 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:39:53.722 7fdaf91a6a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:39:53.722 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:53.722 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:39:53.722 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:53.722 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:53.723 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:39:53.723 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:53.949 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:54.951 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:55.952 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:56.954 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:57.721 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:39:57.955 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:58.957 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:39:59.958 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:00.960 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:01.962 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:02.724 7fdaf91a6a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:40:02.724 7fdaf91a6a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:40:02.724 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:02.724 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:40:02.724 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:02.725 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:40:02.725 7fdaf91a6a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:40:02.725 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:02.725 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:40:02.725 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:02.725 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:02.725 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:40:02.725 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:02.964 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:03.966 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:04.968 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:05.969 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:06.725 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:40:06.970 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:07.972 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:08.974 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:09.975 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:10.976 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:11.727 7fdaf91a6a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:40:11.727 7fdaf91a6a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:40:11.727 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:11.728 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:40:11.728 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:11.728 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:40:11.728 7fdaf91a6a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:40:11.728 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:11.728 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:40:11.729 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:11.729 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:11.729 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:40:11.729 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:11.977 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:12.347 7fdaf91a6a20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:40:12.979 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:13.982 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:14.983 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:15.729 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:40:15.985 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:16.987 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:17.989 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:18.990 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:19.993 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:20.732 7fdaf91a6a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:40:20.732 7fdaf91a6a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:40:20.733 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:20.733 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:40:20.733 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:20.733 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:40:20.733 7fdaf91a6a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:40:20.734 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:20.734 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:40:20.734 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:20.734 7fdaf91a6a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:20.734 7fdaf91a6a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:40:20.734 7fdaf91a6a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:20.994 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:21.996 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:22.769 7fdaf91a6a20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:40:22.997 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:23.998 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:24.733 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:40:25.000 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:26.001 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:27.002 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:28.004 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:29.005 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:29.736 7fdaf91a6b70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:40:29.737 7fdaf91a6b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:40:29.737 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:29.737 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:40:29.737 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:29.737 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:40:29.738 7fdaf91a6b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:40:29.738 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:29.738 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:40:29.738 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:29.738 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:29.738 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:40:29.738 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:30.007 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:31.008 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:32.010 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:33.013 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:33.738 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:40:34.021 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:34.814 7fdaf919ceb8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:40:35.024 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:36.025 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:37.026 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:38.032 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:38.747 7fdaf91a6b70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:40:38.747 7fdaf91a6b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:40:38.747 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:38.747 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:40:38.747 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:38.747 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:40:38.748 7fdaf91a6b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:40:38.748 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:38.748 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:40:38.748 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:38.748 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:38.748 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:40:38.748 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:39.035 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:40.036 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:41.037 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:42.038 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:42.749 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:40:43.040 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:44.042 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:45.043 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:46.044 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:47.045 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:47.751 7fdaf91a6b70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:40:47.751 7fdaf91a6b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:40:47.752 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:47.752 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:40:47.752 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:47.752 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:40:47.752 7fdaf91a6b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:40:47.753 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:47.753 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:40:47.753 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:47.753 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:47.753 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:40:47.753 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:48.047 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:49.048 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:50.049 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:51.051 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:51.752 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:40:52.052 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:53.054 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:54.055 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:55.057 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:56.058 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:56.754 7fdaf91a6b70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:40:56.755 7fdaf91a6b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:40:56.755 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:56.755 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:40:56.755 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:56.755 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:40:56.756 7fdaf91a6b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:40:56.756 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:56.756 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:40:56.756 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:56.756 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:56.756 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:40:56.756 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:57.060 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:58.061 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:40:59.062 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:00.063 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:00.756 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:41:01.065 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:02.067 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:03.069 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:04.070 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:05.072 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:05.759 7fdaf91a6b70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:41:05.759 7fdaf91a6b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:41:05.759 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:05.760 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:41:05.760 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:05.760 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:41:05.761 7fdaf91a6b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:41:05.761 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:05.761 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:41:05.761 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:05.761 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:05.761 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:41:05.762 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:06.073 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:07.075 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:08.076 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:09.077 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:09.760 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:41:10.078 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:11.079 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:12.081 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:13.082 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:14.084 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:14.763 7fdaf91a6b70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:41:14.763 7fdaf91a6b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:41:14.763 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:14.764 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:41:14.764 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:14.764 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:41:14.764 7fdaf91a6b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:41:14.764 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:14.764 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:41:14.765 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:14.765 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:14.765 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:41:14.765 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:15.085 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:16.086 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:17.087 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:18.089 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:18.764 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:41:19.089 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:20.091 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:21.093 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:22.093 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:23.095 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:23.766 7fdaf91a6b70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:41:23.767 7fdaf91a6b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:41:23.767 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:23.767 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:41:23.767 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:23.767 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:41:23.767 7fdaf91a6b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:41:23.767 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:23.768 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:41:23.768 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:23.768 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:23.768 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:41:23.768 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:24.096 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:25.098 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:26.099 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:27.101 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:27.767 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:41:28.102 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:29.104 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:30.105 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:31.107 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:32.108 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:32.769 7fdaf91a6b70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:41:32.769 7fdaf91a6b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:41:32.769 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:32.769 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:41:32.769 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:32.769 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:41:32.770 7fdaf91a6b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:41:32.770 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:32.770 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:41:32.770 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:32.770 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:32.770 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:41:32.770 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:33.109 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:34.111 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:35.112 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:36.114 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:36.770 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:41:37.115 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:38.117 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:39.118 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:40.119 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:41.121 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:41.772 7fdaf91a6b70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:41:41.773 7fdaf91a6b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:41:41.773 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:41.773 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:41:41.773 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:41.773 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:41:41.774 7fdaf91a6b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:41:41.774 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:41.774 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:41:41.774 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:41.774 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:41.774 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:41:41.774 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:42.123 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:43.124 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:44.126 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:45.127 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:45.773 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:41:46.129 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:47.130 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:48.132 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:49.133 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:50.134 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:50.776 7fdaf91a6b70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:41:50.777 7fdaf91a6b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:41:50.777 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:50.777 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:41:50.777 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:50.777 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:41:50.777 7fdaf91a6b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:41:50.778 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:50.778 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:41:50.778 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:50.778 7fdaf91a6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:50.779 7fdaf91a6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:41:50.779 7fdaf91a6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:51.135 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:52.137 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:53.138 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:54.140 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:54.777 7fdaf95fe198 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:41:55.141 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:56.142 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:57.144 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:58.144 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:59.146 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:41:59.778 7fdaf95fe198 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:41:59.779 7fdaf95fe198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:41:59.779 7fdaf95fe198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:59.779 7fdaf95fe198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:41:59.779 7fdaf95fe198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:59.779 7fdaf95fe198 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:41:59.780 7fdaf95fe198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:41:59.780 7fdaf95fe198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:59.780 7fdaf95fe198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:41:59.781 7fdaf95fe198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:59.781 7fdaf95fe198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:59.781 7fdaf95fe198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:41:59.781 7fdaf95fe198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:00.147 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:01.148 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:02.150 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:03.152 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:03.780 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:42:04.154 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:05.156 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:06.157 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:07.158 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:08.160 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:08.782 7fdaf97d3240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:42:08.782 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:42:08.782 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:08.782 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:42:08.783 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:08.783 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:42:08.783 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:42:08.783 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:08.783 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:42:08.783 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:08.783 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:08.783 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:42:08.784 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:09.161 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:10.163 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:11.164 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:12.166 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:12.782 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:42:13.167 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:14.168 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:15.170 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:16.171 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:17.171 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:17.785 7fdaf97d3240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:42:17.786 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:42:17.786 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:17.786 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:42:17.786 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:17.786 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:42:17.787 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:42:17.787 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:17.787 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:42:17.787 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:17.787 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:17.787 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:42:17.787 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:18.173 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:19.174 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:20.175 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:21.177 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:21.786 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:42:22.178 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:23.180 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:24.182 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:25.182 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:26.184 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:26.789 7fdaf97d3240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:42:26.789 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:42:26.789 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:26.789 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:42:26.789 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:26.789 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:42:26.789 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:42:26.790 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:26.790 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:42:26.790 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:26.790 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:26.790 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:42:26.790 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:27.186 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:28.187 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:29.188 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:30.190 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:30.790 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:42:31.191 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:32.193 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:33.194 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:34.195 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:35.196 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:35.792 7fdaf97d3240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:42:35.792 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:42:35.792 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:35.793 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:42:35.793 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:35.793 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:42:35.793 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:42:35.794 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:35.794 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:42:35.794 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:35.794 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:35.794 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:42:35.794 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:36.199 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:37.200 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:38.201 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:39.203 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:39.793 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:42:40.204 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:41.206 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:42.207 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:43.208 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:44.210 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:44.795 7fdaf97d3240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:42:44.795 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:42:44.795 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:44.796 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:42:44.796 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:44.796 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:42:44.796 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:42:44.796 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:44.797 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:42:44.797 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:44.797 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:44.797 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:42:44.797 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:45.212 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:46.214 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:47.216 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:48.217 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:48.796 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:42:49.218 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:50.218 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:51.220 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:52.222 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:53.223 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:53.798 7fdaf97d3240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:42:53.798 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:42:53.798 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:53.798 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:42:53.799 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:53.799 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:42:53.799 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:42:53.799 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:53.799 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:42:53.799 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:53.800 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:53.800 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:42:53.800 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:54.224 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:55.226 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:56.227 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:57.228 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:57.799 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:42:58.229 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:42:59.231 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:00.232 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:01.233 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:02.235 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:02.802 7fdaf97d3240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:43:02.802 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:43:02.802 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:02.802 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:43:02.803 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:02.803 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:43:02.803 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:43:02.804 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:02.804 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:43:02.804 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:02.804 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:02.804 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:43:02.804 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:03.237 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:04.238 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:05.239 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:06.241 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:06.803 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:43:07.242 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:08.243 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:09.245 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:10.246 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:11.065 7fdaf90b1b70 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:43:11.247 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:11.805 7fdaf97d3240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:43:11.806 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:43:11.806 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:11.806 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:43:11.806 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:11.806 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:43:11.807 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:43:11.807 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:11.807 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:43:11.807 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:11.807 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:11.807 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:43:11.807 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:12.248 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:13.250 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:14.252 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:15.254 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:15.807 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:43:16.254 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:17.256 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:18.257 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:19.258 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:20.259 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:20.810 7fdaf97d3240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:43:20.810 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:43:20.810 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:20.810 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:43:20.810 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:20.810 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:43:20.811 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:43:20.811 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:20.811 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:43:20.811 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:20.811 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:20.811 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:43:20.811 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:21.260 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:22.262 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:23.264 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:24.265 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:24.811 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:43:25.267 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:26.268 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:27.269 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:28.270 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:29.271 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:29.813 7fdaf97d3240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:43:29.814 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:43:29.814 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:29.814 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:43:29.814 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:29.814 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:43:29.814 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:43:29.814 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:29.815 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:43:29.815 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:29.816 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:29.816 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:43:29.816 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:30.272 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:31.273 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:32.275 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:33.276 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:33.814 7fdaf90b1048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:43:34.277 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:35.279 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:36.281 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:37.282 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:38.284 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:38.816 7fdaf90b1048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:43:38.816 7fdaf90b1048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:43:38.816 7fdaf90b1048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:38.816 7fdaf90b1048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:43:38.817 7fdaf90b1048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:38.817 7fdaf90b1048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:43:38.817 7fdaf90b1048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:43:38.817 7fdaf90b1048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:38.818 7fdaf90b1048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:43:38.818 7fdaf90b1048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:38.818 7fdaf90b1048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:38.818 7fdaf90b1048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:43:38.818 7fdaf90b1048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:39.285 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:40.287 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:41.289 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:42.290 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:42.817 7fdaf90b1048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:43:43.291 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:44.292 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:45.294 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:46.295 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:47.297 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:47.819 7fdaf90b1048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:43:47.819 7fdaf90b1048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:43:47.819 7fdaf90b1048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:47.819 7fdaf90b1048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:43:47.819 7fdaf90b1048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:47.819 7fdaf90b1048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:43:47.820 7fdaf90b1048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:43:47.820 7fdaf90b1048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:47.820 7fdaf90b1048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:43:47.820 7fdaf90b1048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:47.820 7fdaf90b1048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:47.820 7fdaf90b1048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:43:47.820 7fdaf90b1048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:48.298 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:49.300 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:50.302 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:51.303 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:51.819 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:43:52.305 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:53.306 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:54.308 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:55.309 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:56.310 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:56.822 7fdaf97d3240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:43:56.822 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:43:56.823 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:56.823 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:43:56.823 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:56.823 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:43:56.823 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:43:56.824 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:56.824 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:43:56.824 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:56.824 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:56.824 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:43:56.824 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:57.311 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:58.312 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:43:59.314 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:00.316 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:00.823 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:44:01.318 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:02.319 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:03.321 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:04.322 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:05.323 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:05.826 7fdaf97d3240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:44:05.826 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:44:05.826 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:05.826 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:44:05.826 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:05.826 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:44:05.827 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:44:05.827 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:05.827 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:44:05.827 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:05.827 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:05.827 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:44:05.827 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:06.325 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:07.327 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:08.328 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:09.330 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:09.826 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:44:10.331 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:11.333 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:12.334 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:13.335 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:14.336 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:14.828 7fdaf97d3240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:44:14.829 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:44:14.829 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:14.829 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:44:14.829 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:14.829 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:44:14.830 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:44:14.830 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:14.830 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:44:14.830 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:14.830 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:14.830 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:44:14.830 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:15.338 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:16.339 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:17.340 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:18.342 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:18.829 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:44:19.343 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:20.345 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:21.347 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:22.348 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:23.350 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:23.830 7fdaf97d3240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:44:23.831 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:44:23.831 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:23.831 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:44:23.831 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:23.831 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:44:23.832 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:44:23.832 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:23.832 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:44:23.832 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:23.832 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:23.832 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:44:23.832 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:24.351 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:25.353 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:26.354 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:27.354 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:27.832 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:44:28.356 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:29.357 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:30.359 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:31.360 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:32.361 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:32.834 7fdaf97d3240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:44:32.835 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:44:32.835 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:32.835 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:44:32.835 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:32.836 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:44:32.836 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:44:32.836 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:32.837 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:44:32.837 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:32.838 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:32.838 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:44:32.838 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:33.362 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:34.363 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:35.364 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:36.365 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:36.834 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:44:37.367 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:38.368 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:39.370 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:40.371 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:41.372 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:41.837 7fdaf97d3240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:44:41.838 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:44:41.838 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:41.838 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:44:41.838 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:41.838 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:44:41.839 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:44:41.840 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:41.840 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:44:41.840 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:41.840 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:41.840 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:44:41.840 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:42.374 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:43.376 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:44.378 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:45.379 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:45.838 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:44:46.381 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:47.382 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:48.383 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:49.384 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:50.386 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:50.841 7fdaf97d3240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:44:50.841 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:44:50.841 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:50.841 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:44:50.841 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:50.841 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:44:50.842 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:44:50.842 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:50.842 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:44:50.842 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:50.842 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:50.842 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:44:50.842 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:51.387 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:52.387 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:53.389 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:54.390 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:54.842 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:44:55.391 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:56.393 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:57.394 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:58.396 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:59.397 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:44:59.844 7fdaf97d3240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:44:59.844 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:44:59.844 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:59.845 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:44:59.845 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:59.845 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:44:59.845 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:44:59.845 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:59.845 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:44:59.845 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:59.846 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:59.846 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:44:59.846 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:00.398 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:01.400 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:02.401 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:03.403 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:03.845 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:45:04.404 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:05.405 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:06.406 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:07.407 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:08.409 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:08.848 7fdaf97d3240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:45:08.848 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:45:08.848 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:08.849 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:45:08.850 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:08.850 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:45:08.850 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:45:08.850 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:08.850 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:45:08.850 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:08.850 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:08.851 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:45:08.851 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:09.410 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:10.412 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:11.413 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:12.414 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:12.849 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:45:13.416 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:14.418 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:15.419 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:16.421 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:17.422 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:17.851 7fdaf97d3240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:45:17.852 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:45:17.852 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:17.852 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:45:17.852 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:17.852 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:45:17.853 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:45:17.853 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:17.853 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:45:17.853 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:17.854 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:17.854 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:45:17.854 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:18.424 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:19.426 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:20.428 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:21.429 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:21.853 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:45:22.430 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:23.433 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:24.445 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:25.446 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:26.448 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:26.854 7fdaf97d3240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:45:26.855 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:45:26.855 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:26.855 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:45:26.855 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:26.855 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:45:26.856 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:45:26.856 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:26.856 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:45:26.856 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:26.856 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:26.856 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:45:26.856 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:27.449 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:28.450 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:29.452 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:30.453 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:30.855 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:45:31.455 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:32.457 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:33.459 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:34.461 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:35.463 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:35.856 7fdaf97d3240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:45:35.857 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:45:35.857 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:35.857 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:45:35.857 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:35.857 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:45:35.858 7fdaf97d3240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:45:35.858 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:35.858 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:45:35.858 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:35.858 7fdaf97d3240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:35.858 7fdaf97d3240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:45:35.858 7fdaf97d3240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:36.465 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:37.473 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:38.474 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:39.476 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:39.857 7fdaf90770f0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:45:40.477 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:41.479 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:42.481 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:43.482 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:44.483 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:44.859 7fdaf90770f0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:45:44.859 7fdaf90770f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:45:44.859 7fdaf90770f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:44.860 7fdaf90770f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:45:44.860 7fdaf90770f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:44.860 7fdaf90770f0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:45:44.860 7fdaf90770f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:45:44.860 7fdaf90770f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:44.860 7fdaf90770f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:45:44.860 7fdaf90770f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:44.861 7fdaf90770f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:44.861 7fdaf90770f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:45:44.861 7fdaf90770f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:45.485 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:46.486 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:47.488 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:48.489 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:48.860 7fdaf90770f0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:45:49.491 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:50.178 7fdaf9766a20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:45:50.492 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:51.493 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:52.494 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:53.495 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:53.862 7fdaf90770f0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:45:53.863 7fdaf90770f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:45:53.863 7fdaf90770f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:53.863 7fdaf90770f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:45:53.863 7fdaf90770f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:53.863 7fdaf90770f0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:45:53.864 7fdaf90770f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:45:53.864 7fdaf90770f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:53.864 7fdaf90770f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:45:53.864 7fdaf90770f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:53.864 7fdaf90770f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:53.864 7fdaf90770f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:45:53.864 7fdaf90770f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:54.497 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:55.499 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:56.500 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:57.501 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:57.863 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:45:58.502 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:45:59.503 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:00.253 7fdaf9077438 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:46:00.504 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:01.505 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:02.507 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:02.865 7fdaf9766a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:46:02.866 7fdaf9766a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:46:02.866 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:02.866 7fdaf9766a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:46:02.866 7fdaf9766a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:02.866 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:46:02.867 7fdaf9766a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:46:02.867 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:02.867 7fdaf9766a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:46:02.867 7fdaf9766a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:02.867 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:02.867 7fdaf9766a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:46:02.867 7fdaf9766a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:03.509 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:04.511 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:05.513 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:06.514 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:06.866 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:46:07.516 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:08.517 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:09.518 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:10.519 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:11.183 7fdaf902a0f0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:46:11.520 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:11.868 7fdaf9766a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:46:11.868 7fdaf9766a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:46:11.868 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:11.868 7fdaf9766a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:46:11.869 7fdaf9766a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:11.869 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:46:11.869 7fdaf9766a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:46:11.869 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:11.869 7fdaf9766a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:46:11.869 7fdaf9766a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:11.870 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:11.870 7fdaf9766a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:46:11.870 7fdaf9766a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:12.521 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:13.522 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:14.524 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:15.525 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:15.868 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:46:16.526 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:17.527 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:18.529 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:19.529 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:20.531 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:20.871 7fdaf9766a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:46:20.871 7fdaf9766a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:46:20.871 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:20.871 7fdaf9766a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:46:20.871 7fdaf9766a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:20.871 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:46:20.872 7fdaf9766a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:46:20.872 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:20.872 7fdaf9766a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:46:20.872 7fdaf9766a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:20.872 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:20.872 7fdaf9766a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:46:20.872 7fdaf9766a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:21.532 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:22.534 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:23.536 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:24.539 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:24.871 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:46:25.540 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:26.541 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:27.543 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:28.545 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:29.546 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:29.874 7fdaf9766a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:46:29.874 7fdaf9766a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:46:29.874 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:29.874 7fdaf9766a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:46:29.874 7fdaf9766a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:29.874 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:46:29.875 7fdaf9766a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:46:29.875 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:29.875 7fdaf9766a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:46:29.875 7fdaf9766a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:29.875 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:29.875 7fdaf9766a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:46:29.875 7fdaf9766a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:30.548 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:31.549 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:32.550 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:33.552 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:33.875 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:46:34.553 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:35.554 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:36.557 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:37.558 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:38.560 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:38.878 7fdaf9766a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:46:38.878 7fdaf9766a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:46:38.878 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:38.878 7fdaf9766a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:46:38.878 7fdaf9766a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:38.878 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:46:38.879 7fdaf9766a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:46:38.879 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:38.879 7fdaf9766a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:46:38.879 7fdaf9766a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:38.879 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:38.879 7fdaf9766a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:46:38.879 7fdaf9766a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:39.561 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:40.562 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:41.564 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:42.565 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:42.878 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:46:43.567 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:44.569 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:45.570 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:46.572 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:47.573 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:47.880 7fdaf9766a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:46:47.881 7fdaf9766a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:46:47.881 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:47.881 7fdaf9766a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:46:47.881 7fdaf9766a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:47.881 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:46:47.882 7fdaf9766a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:46:47.882 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:47.882 7fdaf9766a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:46:47.882 7fdaf9766a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:47.882 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:47.882 7fdaf9766a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:46:47.882 7fdaf9766a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:48.575 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:49.577 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:50.579 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:51.581 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:51.881 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:46:52.582 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:53.584 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:54.586 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:55.587 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:56.588 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:56.884 7fdaf9766a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:46:56.885 7fdaf9766a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:46:56.885 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:56.885 7fdaf9766a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:46:56.885 7fdaf9766a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:56.885 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:46:56.886 7fdaf9766a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:46:56.886 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:56.886 7fdaf9766a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:46:56.886 7fdaf9766a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:56.886 7fdaf9766a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:56.886 7fdaf9766a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:46:56.886 7fdaf9766a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:57.589 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:58.591 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:46:59.593 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:00.594 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:00.885 7fdaf95840f0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:47:01.596 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:02.597 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:03.598 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:04.599 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:05.600 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:05.887 7fdaf95840f0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:47:05.888 7fdaf95840f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:47:05.888 7fdaf95840f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:05.888 7fdaf95840f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:47:05.888 7fdaf95840f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:05.888 7fdaf95840f0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:47:05.889 7fdaf95840f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:47:05.889 7fdaf95840f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:05.889 7fdaf95840f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:47:05.889 7fdaf95840f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:05.889 7fdaf95840f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:05.889 7fdaf95840f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:47:05.889 7fdaf95840f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:06.602 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:07.604 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:08.606 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:09.607 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:09.888 7fdaf95840f0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:47:10.609 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:11.610 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:12.611 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:13.612 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:14.614 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:14.891 7fdaf95840f0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:47:14.891 7fdaf95840f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:47:14.891 7fdaf95840f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:14.891 7fdaf95840f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:47:14.891 7fdaf95840f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:14.891 7fdaf95840f0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:47:14.892 7fdaf95840f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:47:14.892 7fdaf95840f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:14.893 7fdaf95840f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:47:14.893 7fdaf95840f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:14.893 7fdaf95840f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:14.893 7fdaf95840f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:47:14.893 7fdaf95840f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:15.616 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:16.618 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:17.619 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:18.621 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:18.891 7fdaf95840f0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:47:19.622 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:20.623 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:21.625 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:21.630 7fdaf957ce10 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:47:22.626 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:23.628 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:23.894 7fdaf95840f0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:47:23.894 7fdaf95840f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:47:23.894 7fdaf95840f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:23.894 7fdaf95840f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:47:23.894 7fdaf95840f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:23.894 7fdaf95840f0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:47:23.895 7fdaf95840f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:47:23.895 7fdaf95840f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:23.895 7fdaf95840f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:47:23.895 7fdaf95840f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:23.895 7fdaf95840f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:23.895 7fdaf95840f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:47:23.895 7fdaf95840f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:24.630 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:25.632 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:26.633 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:27.634 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:27.894 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:47:28.638 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:29.640 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:30.641 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:31.643 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:32.644 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:32.896 7fdaf2b21978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:47:32.896 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:47:32.896 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:32.897 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:47:32.897 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:32.897 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:47:32.897 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:47:32.897 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:32.897 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:47:32.897 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:32.897 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:32.898 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:47:32.898 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:33.647 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:34.648 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:35.650 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:36.652 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:36.897 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:47:37.653 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:38.654 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:39.655 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:40.658 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:41.659 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:41.899 7fdaf2b21978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:47:41.899 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:47:41.899 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:41.899 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:47:41.900 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:41.900 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:47:41.900 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:47:41.900 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:41.901 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:47:41.901 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:41.901 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:41.901 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:47:41.901 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:42.660 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:43.662 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:44.663 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:45.664 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:45.899 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:47:46.666 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:47.668 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:48.669 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:49.670 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:50.671 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:50.901 7fdaf2b21978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:47:50.901 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:47:50.901 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:50.901 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:47:50.901 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:50.901 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:47:50.902 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:47:50.902 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:50.902 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:47:50.902 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:50.902 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:50.902 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:47:50.902 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:51.673 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:52.675 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:53.675 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:54.677 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:54.901 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:47:55.679 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:56.681 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:57.682 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:58.684 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:59.686 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:47:59.904 7fdaf2b21978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:47:59.904 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:47:59.904 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:59.904 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:47:59.904 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:59.905 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:47:59.905 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:47:59.905 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:59.905 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:47:59.905 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:59.905 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:59.905 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:47:59.906 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:00.687 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:01.688 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:02.689 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:03.691 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:03.904 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:48:04.692 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:05.694 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:06.694 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:07.696 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:08.697 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:08.906 7fdaf2b21978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:48:08.907 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:48:08.907 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:48:08.907 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:48:08.907 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:48:08.907 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:48:08.907 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:48:08.908 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:08.908 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:48:08.908 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:08.908 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:08.908 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:48:08.908 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:09.699 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:10.701 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:11.702 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:12.704 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:12.906 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:48:13.705 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:14.707 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:15.708 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:16.709 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:17.710 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:17.909 7fdaf2b21978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:48:17.909 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:48:17.909 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:48:17.909 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:48:17.909 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:48:17.909 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:48:17.910 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:48:17.911 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:17.911 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:48:17.911 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:17.912 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:17.912 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:48:17.912 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:18.712 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:19.714 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:20.715 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:21.717 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:21.909 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:48:22.719 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:23.480 7fdaf90e64e0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:48:23.720 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:24.721 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:25.723 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:26.725 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:26.911 7fdaf2b21978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:48:26.912 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:48:26.912 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:48:26.912 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:48:26.912 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:48:26.912 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:48:26.913 7fdaf2b21978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:48:26.913 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:26.913 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.129:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.129:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.129:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.129:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:48:26.913 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:26.913 7fdaf2b21978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:26.914 7fdaf2b21978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 119 http_port = 8096 start_time = 1748830048869418 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 10 >> 06/02/2025 02:48:26.914 7fdaf2b21978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:27.727 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:28.728 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 06/02/2025 02:48:29.729 7fdaf97d3d68 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process.