11/07/2024 02:11:44.094 7fafea104988 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 11/07/2024 02:11:44.094 7fafea104988 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 11/07/2024 02:11:44.095 7fafea104988 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 11/07/2024 02:11:44.095 7fafe16a46d8 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 11/07/2024 02:11:44.096 7fafe16a46d8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 11/07/2024 02:11:44.096 7fafe16a46d8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/07/2024 02:11:44.097 7fafe16a46d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 11/07/2024 02:11:44.097 7fafe16a46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:44.098 7fafe16a46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:11:44.098 7fafe16a46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:44.098 7fafe16a46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:44.098 7fafe16a46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 start_time = 1730945504096687 collector_name = collector_ip = collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 11/07/2024 02:11:44.098 7fafe16a46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:44.162 7fafea104988 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.35:5673//'] 11/07/2024 02:11:44.175 7fafea104988 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 11/07/2024 02:11:44.176 7fafe1676630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:44.176 7fafe1676630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/07/2024 02:11:44.177 7fafe1676630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:44.177 7fafe1676630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:44.178 7fafe1676630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/07/2024 02:11:44.178 7fafe1676630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:44.181 7fafea104988 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 11/07/2024 02:11:44.182 7fafe163deb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:44.182 7fafe163deb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:11:44.182 7fafe163deb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:44.203 7fafea104988 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 11/07/2024 02:11:44.204 7fafe1630eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:44.205 7fafe1630eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/07/2024 02:11:44.205 7fafe1630eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:44.211 7fafea104988 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 11/07/2024 02:11:44.211 7fafe1630eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:44.211 7fafe1630eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:11:44.211 7fafe1630eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:44.217 7fafea104988 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 11/07/2024 02:11:44.218 7fafea104988 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 11/07/2024 02:11:44.219 7fafea104988 [contrail-device-manager] [WARNING]: initializing cassandra cluster 11/07/2024 02:11:44.219 7fafea104988 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 11/07/2024 02:11:44.220 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 11/07/2024 02:11:44.220 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:11:44.220 7fafe163df60 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 11/07/2024 02:11:44.220 7fafe163df60 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 11/07/2024 02:11:44.222 7fafe1630390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:44.222 7fafe1630390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/07/2024 02:11:44.223 7fafe1630390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:44.224 7fafea104988 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 11/07/2024 02:11:44.228 7fafe6f94780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:11:44.229 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 11/07/2024 02:11:44.230 7fafe163df60 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 11/07/2024 02:11:44.252 7fafea104988 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 11/07/2024 02:11:44.254 7fafe0bc3978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:44.254 7fafe0bc3978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 11/07/2024 02:11:44.255 7fafe0bc3978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:44.262 7fafea104988 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 11/07/2024 02:11:44.262 7fafe14bb828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:44.262 7fafe14bb828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/07/2024 02:11:44.262 7fafe14bb828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:44.267 7fafea104988 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 11/07/2024 02:11:44.277 7fafea104988 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 11/07/2024 02:11:44.321 7fafea104988 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/07/2024 02:11:44.420 7fafea104988 [contrail-device-manager] [WARNING]: table 'obj_shared_table', created 11/07/2024 02:11:44.425 7fafea104988 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/07/2024 02:11:44.430 7fafea104988 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/07/2024 02:11:44.505 7fafea104988 [contrail-device-manager] [WARNING]: table 'obj_shared_table' fixed with properties {'gc_grace_seconds': 864000} 11/07/2024 02:11:44.508 7fafea104988 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/07/2024 02:11:44.508 7fafe14bba20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:44.508 7fafe14bba20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:11:44.508 7fafe14bba20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:44.516 7fafea104988 [contrail-device-manager] [ERROR]: Error in db walk read 'type' 11/07/2024 02:11:44.518 7fafe0bc8b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:44.519 7fafe0bc8b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:Zookeeper[] connection down >>, ] >> 11/07/2024 02:11:44.519 7fafe0bc8b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:44.525 7fafe0bc7780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:44.525 7fafe0bc7780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:11:44.526 7fafe0bc7780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:44.526 7fafea104988 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 11/07/2024 02:11:44.526 7fafea104988 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 11/07/2024 02:11:44.527 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:11:44.527 7fafea104988 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1. 11/07/2024 02:11:44.530 7fafe6f94780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:11:44.538 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.'] 11/07/2024 02:11:44.545 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:11:44.548 7fafe6f94780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:11:44.553 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.'] 11/07/2024 02:11:45.039 7fafea104988 [contrail-device-manager] [DEBUG]: Fetching all containers 11/07/2024 02:11:45.182 7fafea104988 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 11/07/2024 02:11:45.395 7fafea104988 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..config_queue 11/07/2024 02:11:45.395 7fafea104988 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..tftp_queue 11/07/2024 02:11:45.557 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:11:45.563 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:11:45.567 7fafe6f94780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:11:45.581 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..tftp_queue'] 11/07/2024 02:11:45.896 7fafea104988 [contrail-device-manager] [WARNING]: Process 114 prepared to run in Full mode... 11/07/2024 02:11:45.896 7fafea104988 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 11/07/2024 02:11:45.896 7fafea104988 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 11/07/2024 02:11:45.899 7fafea104988 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page 11/07/2024 02:11:45.899 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 11/07/2024 02:11:45.900 7fafea104988 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page 11/07/2024 02:11:45.900 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 11/07/2024 02:11:45.901 7fafea104988 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 11/07/2024 02:11:45.901 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 11/07/2024 02:11:45.908 7fafea104988 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page 11/07/2024 02:11:45.908 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 11/07/2024 02:11:45.908 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 11/07/2024 02:11:45.908 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 11/07/2024 02:11:45.908 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 11/07/2024 02:11:45.908 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 11/07/2024 02:11:45.908 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 11/07/2024 02:11:45.908 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 11/07/2024 02:11:45.908 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 11/07/2024 02:11:45.908 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 11/07/2024 02:11:45.909 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 11/07/2024 02:11:45.909 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 11/07/2024 02:11:45.909 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 11/07/2024 02:11:45.910 7fafea104988 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 11/07/2024 02:11:45.910 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 11/07/2024 02:11:45.910 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 11/07/2024 02:11:45.910 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 11/07/2024 02:11:45.911 7fafea104988 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 11/07/2024 02:11:45.911 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 11/07/2024 02:11:45.911 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 11/07/2024 02:11:45.913 7fafea104988 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page 11/07/2024 02:11:45.913 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 11/07/2024 02:11:45.914 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 11/07/2024 02:11:45.914 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 11/07/2024 02:11:45.926 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 11/07/2024 02:11:45.926 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 11/07/2024 02:11:45.927 7fafea104988 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page 11/07/2024 02:11:45.927 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 11/07/2024 02:11:45.927 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 11/07/2024 02:11:45.927 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 11/07/2024 02:11:45.927 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 11/07/2024 02:11:45.927 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 11/07/2024 02:11:45.928 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 11/07/2024 02:11:45.928 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 11/07/2024 02:11:45.928 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 11/07/2024 02:11:45.929 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 11/07/2024 02:11:45.929 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 11/07/2024 02:11:45.929 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 11/07/2024 02:11:45.929 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 11/07/2024 02:11:45.929 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 11/07/2024 02:11:45.929 7fafea104988 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 11/07/2024 02:11:45.932 7fafea104988 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 11/07/2024 02:11:45.990 7fafe0af6a20 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 11/07/2024 02:11:45.990 7fafe0af6a20 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 11/07/2024 02:11:45.990 7fafe02d6780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:45.990 7fafe02d6780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[none to Idle on EvStart], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 11/07/2024 02:11:45.990 7fafe02d6780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:45.995 7fafe0a07c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:45.995 7fafe0a07c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[none to Idle on EvStart], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 11/07/2024 02:11:45.995 7fafe0a07c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:46.583 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:11:46.591 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:11:46.594 7fafe6f94780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:11:46.607 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..tftp_queue', 'device_manager_ztp.ztp_queue'] 11/07/2024 02:11:47.610 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:11:48.097 7fafe0224a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:11:48.098 7fafe0224a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:11:48.098 7fafe0224a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:11:48.098 7fafe0224a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:11:48.099 7fafe0224a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 11/07/2024 02:11:48.099 7fafe0224a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:11:48.099 7fafe0224a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:11:48.099 7fafe0224a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:11:48.099 7fafe0224a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:48.099 7fafe0224a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 11/07/2024 02:11:48.099 7fafe0224a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:48.099 7fafe0224a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:48.100 7fafe0224a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945504096687 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:11:48.100 7fafe0224a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:48.612 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:11:49.118 7fafe0216b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:49.118 7fafe0216b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:11:49.118 7fafe0216b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:49.613 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:11:50.615 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:11:51.615 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:11:52.099 7fafe0143eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:11:52.100 7fafe0143eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:11:52.100 7fafe0143eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:11:52.100 7fafe0143eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:11:52.100 7fafe0143eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:11:52.100 7fafe0143eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:11:52.100 7fafe0143eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:11:52.101 7fafe0143eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:11:52.101 7fafe0143eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:52.101 7fafe0143eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:11:52.101 7fafe0143eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:52.101 7fafe0143eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:52.101 7fafe0143eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945504096687 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:11:52.101 7fafe0143eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:52.616 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:11:53.617 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:11:54.618 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:11:55.619 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:11:56.101 7fafe017cf60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:11:56.101 7fafe017cf60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:11:56.102 7fafe017cf60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:11:56.102 7fafe017cf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:11:56.102 7fafe017cf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:11:56.102 7fafe017cf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:11:56.102 7fafe017cf60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:11:56.102 7fafe017cf60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:11:56.102 7fafe017cf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:56.102 7fafe017cf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:11:56.103 7fafe017cf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:56.103 7fafe017cf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:56.103 7fafe017cf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945504096687 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:11:56.103 7fafe017cf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:11:56.620 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:11:57.621 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:11:58.621 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:11:59.622 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:00.102 7fafe022df60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:12:00.104 7fafe022df60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:12:00.104 7fafe022df60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:12:00.104 7fafe022df60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:00.104 7fafe022df60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:12:00.104 7fafe022df60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:00.104 7fafe022df60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:12:00.104 7fafe022df60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:12:00.105 7fafe022df60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:00.105 7fafe022df60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:12:00.105 7fafe022df60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:00.105 7fafe022df60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:00.105 7fafe022df60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945504096687 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:12:00.105 7fafe022df60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:00.623 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:01.625 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:02.625 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:03.625 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:04.104 7fafe0212cc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:12:04.105 7fafe0212cc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:12:04.105 7fafe0212cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:12:04.105 7fafe0212cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:04.105 7fafe0212cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:12:04.105 7fafe0212cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:04.105 7fafe0212cc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:12:04.106 7fafe0212cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:12:04.106 7fafe0212cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:04.106 7fafe0212cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:12:04.106 7fafe0212cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:04.106 7fafe0212cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:04.106 7fafe0212cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945504096687 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:12:04.106 7fafe0212cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:04.626 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:05.627 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:06.628 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:07.629 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:08.106 7fafe0212978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:12:08.107 7fafe0212978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:12:08.107 7fafe0212978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:12:08.107 7fafe0212978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:08.107 7fafe0212978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:12:08.107 7fafe0212978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:08.107 7fafe0212978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:12:08.108 7fafe0212978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:12:08.108 7fafe0212978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:08.108 7fafe0212978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:12:08.108 7fafe0212978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:08.108 7fafe0212978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:08.108 7fafe0212978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945504096687 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:12:08.108 7fafe0212978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:08.630 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:09.631 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:10.633 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:11.633 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:12.107 7fafe710c240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:12:12.109 7fafe710c240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:12:12.109 7fafe710c240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:12:12.109 7fafe710c240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:12.109 7fafe710c240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:12:12.109 7fafe710c240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:12.109 7fafe710c240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:12:12.110 7fafe710c240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:12:12.110 7fafe710c240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:12.110 7fafe710c240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:12:12.110 7fafe710c240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:12.110 7fafe710c240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:12.110 7fafe710c240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945504096687 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:12:12.110 7fafe710c240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:12.635 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:13.636 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:14.637 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:15.638 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:16.109 7fafe0229198 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:12:16.110 7fafe0229198 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:12:16.110 7fafe0229198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:12:16.110 7fafe0229198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:16.111 7fafe0229198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:12:16.111 7fafe0229198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:16.111 7fafe0229198 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:12:16.111 7fafe0229198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:12:16.111 7fafe0229198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:16.111 7fafe0229198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:12:16.111 7fafe0229198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:16.112 7fafe0229198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:16.112 7fafe0229198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945504096687 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:12:16.112 7fafe0229198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:16.639 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:17.641 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:18.642 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:19.643 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:20.111 7fafe70fcc18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:12:20.112 7fafe70fcc18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:12:20.112 7fafe70fcc18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:12:20.112 7fafe70fcc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:20.112 7fafe70fcc18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:12:20.112 7fafe70fcc18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:20.112 7fafe70fcc18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:12:20.113 7fafe70fcc18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:12:20.113 7fafe70fcc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:20.113 7fafe70fcc18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:12:20.113 7fafe70fcc18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:20.113 7fafe70fcc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:20.113 7fafe70fcc18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945504096687 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:12:20.113 7fafe70fcc18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:20.644 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:21.645 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:22.646 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:23.647 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:24.113 7fafe0147978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:12:24.114 7fafe0147978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:12:24.114 7fafe0147978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:12:24.114 7fafe0147978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:24.114 7fafe0147978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:12:24.114 7fafe0147978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:24.114 7fafe0147978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:12:24.115 7fafe0147978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:12:24.115 7fafe0147978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:24.115 7fafe0147978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:12:24.115 7fafe0147978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:24.115 7fafe0147978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:24.115 7fafe0147978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945504096687 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:12:24.115 7fafe0147978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:24.648 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:25.648 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:26.649 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:27.651 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:28.115 7fafe0147630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:12:28.116 7fafe0147630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:12:28.116 7fafe0147630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:12:28.116 7fafe0147630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:28.116 7fafe0147630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:12:28.116 7fafe0147630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:28.116 7fafe0147630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:12:28.117 7fafe0147630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:12:28.117 7fafe0147630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:28.117 7fafe0147630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:12:28.117 7fafe0147630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:28.117 7fafe0147630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:28.117 7fafe0147630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945504096687 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:12:28.117 7fafe0147630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:28.651 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:29.653 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:30.653 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:31.654 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:32.116 7fafe0147780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:12:32.117 7fafe0147780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:12:32.117 7fafe0147780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:12:32.117 7fafe0147780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:32.117 7fafe0147780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:12:32.117 7fafe0147780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:32.117 7fafe0147780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:12:32.118 7fafe0147780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:12:32.118 7fafe0147780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:32.118 7fafe0147780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:12:32.118 7fafe0147780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:32.118 7fafe0147780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:32.118 7fafe0147780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945504096687 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:12:32.118 7fafe0147780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:32.655 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:33.657 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:34.658 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:35.659 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:36.117 7fafe716d240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:12:36.119 7fafe716d240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:12:36.119 7fafe716d240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:12:36.119 7fafe716d240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:36.119 7fafe716d240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:12:36.119 7fafe716d240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:36.119 7fafe716d240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:12:36.119 7fafe716d240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:12:36.120 7fafe716d240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:36.120 7fafe716d240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:12:36.120 7fafe716d240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:36.120 7fafe716d240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:36.120 7fafe716d240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945504096687 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:12:36.120 7fafe716d240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:36.660 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:37.660 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:38.662 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:39.662 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:40.120 7fafe0232048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:12:40.121 7fafe0232048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:12:40.121 7fafe0232048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:12:40.121 7fafe0232048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:40.121 7fafe0232048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:12:40.121 7fafe0232048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:40.121 7fafe0232048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:12:40.121 7fafe0232048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:12:40.121 7fafe0232048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:40.122 7fafe0232048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:12:40.122 7fafe0232048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:40.122 7fafe0232048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:40.122 7fafe0232048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945504096687 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:12:40.122 7fafe0232048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:40.664 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:41.665 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:42.666 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:43.668 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:44.121 7fafe14c0a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:12:44.122 7fafe14c0a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:12:44.122 7fafe14c0a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:12:44.122 7fafe14c0a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:44.122 7fafe14c0a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:12:44.122 7fafe14c0a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:44.123 7fafe14c0a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:12:44.123 7fafe14c0a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:12:44.123 7fafe14c0a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:44.123 7fafe14c0a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:12:44.123 7fafe14c0a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:44.123 7fafe14c0a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:44.123 7fafe14c0a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945504096687 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:12:44.123 7fafe14c0a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:44.669 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:45.670 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:46.671 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:47.672 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:48.123 7fafe0226390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:12:48.124 7fafe0226390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:12:48.124 7fafe0226390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:12:48.124 7fafe0226390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:48.124 7fafe0226390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:12:48.124 7fafe0226390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:48.124 7fafe0226390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:12:48.124 7fafe0226390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:12:48.125 7fafe0226390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:48.125 7fafe0226390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:12:48.125 7fafe0226390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:48.125 7fafe0226390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:48.125 7fafe0226390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945504096687 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:12:48.125 7fafe0226390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:48.673 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:49.673 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:50.674 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:51.675 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:12:52.124 7fafe0143c18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:12:52.125 7fafe0143c18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:12:52.125 7fafe0143c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:12:52.125 7fafe0143c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:52.126 7fafe0143c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:12:52.126 7fafe0143c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:12:52.126 7fafe0143c18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:12:52.126 7fafe0143c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:12:52.126 7fafe0143c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:52.126 7fafe0143c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:12:52.126 7fafe0143c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:52.126 7fafe0143c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:52.126 7fafe0143c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945504096687 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:12:52.126 7fafe0143c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:12:52.676 7fafe6f94780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:13:44.532 7f4d0bb66f48 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 11/07/2024 02:13:44.532 7f4d0bb66f48 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 11/07/2024 02:13:44.532 7f4d0bb66f48 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 11/07/2024 02:13:44.532 7f4d031d76d8 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 11/07/2024 02:13:44.533 7f4d031d76d8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 11/07/2024 02:13:44.533 7f4d031d76d8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/07/2024 02:13:44.533 7f4d031d76d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 11/07/2024 02:13:44.533 7f4d031d76d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:44.533 7f4d031d76d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:13:44.533 7f4d031d76d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:44.533 7f4d031d76d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:44.533 7f4d031d76d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 start_time = 1730945624533267 collector_name = collector_ip = collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 11/07/2024 02:13:44.533 7f4d031d76d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:44.568 7f4d0bb66f48 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.35:5673//'] 11/07/2024 02:13:44.572 7f4d0bb66f48 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 11/07/2024 02:13:44.573 7f4d03129240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:44.573 7f4d03129240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/07/2024 02:13:44.573 7f4d03129240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:44.573 7f4d03129240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:44.573 7f4d03129240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/07/2024 02:13:44.573 7f4d03129240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:44.576 7f4d0bb66f48 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 11/07/2024 02:13:44.576 7f4d031290f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:44.577 7f4d031290f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:13:44.577 7f4d031290f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:44.582 7f4d0bb66f48 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 11/07/2024 02:13:44.583 7f4d030822e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:44.583 7f4d030822e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/07/2024 02:13:44.583 7f4d030822e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:44.585 7f4d0bb66f48 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 11/07/2024 02:13:44.586 7f4d030822e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:44.586 7f4d030822e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:13:44.586 7f4d030822e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:44.596 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 11/07/2024 02:13:44.596 7f4d0bb66f48 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 11/07/2024 02:13:44.596 7f4d0bb66f48 [contrail-device-manager] [WARNING]: initializing cassandra cluster 11/07/2024 02:13:44.597 7f4d0bb66f48 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 11/07/2024 02:13:44.597 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 11/07/2024 02:13:44.597 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:13:44.597 7f4d05bed0f0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 11/07/2024 02:13:44.598 7f4d05bed0f0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 11/07/2024 02:13:44.598 7f4d03082cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:44.598 7f4d03082cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/07/2024 02:13:44.598 7f4d03082cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:44.599 7f4d0bb66f48 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 11/07/2024 02:13:44.606 7f4d0b7a76d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:13:44.606 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 11/07/2024 02:13:44.606 7f4d05bed0f0 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 11/07/2024 02:13:44.621 7f4d0bb66f48 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 11/07/2024 02:13:44.623 7f4d027238d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:44.623 7f4d027238d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 11/07/2024 02:13:44.623 7f4d027238d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:44.627 7f4d0bb66f48 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 11/07/2024 02:13:44.628 7f4d02728828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:44.628 7f4d02728828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/07/2024 02:13:44.628 7f4d02728828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:44.631 7f4d0bb66f48 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 11/07/2024 02:13:44.639 7f4d0bb66f48 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 11/07/2024 02:13:44.640 7f4d027382e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:44.640 7f4d027382e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:13:44.640 7f4d027382e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:44.654 7f4d0302ad68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:44.654 7f4d0302ad68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:Zookeeper[] connection down >>, ] >> 11/07/2024 02:13:44.654 7f4d0302ad68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:44.661 7f4d02735e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:44.662 7f4d02735e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:13:44.662 7f4d02735e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:44.662 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 11/07/2024 02:13:44.662 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 11/07/2024 02:13:44.663 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:13:44.663 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1. 11/07/2024 02:13:44.666 7f4d0b7a76d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:13:44.671 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.'] 11/07/2024 02:13:44.681 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:13:44.685 7f4d0b7a76d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:13:44.689 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.'] 11/07/2024 02:13:45.175 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Fetching all containers 11/07/2024 02:13:45.335 7f4d0bb66f48 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 11/07/2024 02:13:45.515 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..config_queue 11/07/2024 02:13:45.515 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..tftp_queue 11/07/2024 02:13:45.691 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:13:45.699 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:13:45.703 7f4d0b7a76d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:13:45.712 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..tftp_queue'] 11/07/2024 02:13:46.017 7f4d0bb66f48 [contrail-device-manager] [WARNING]: Process 114 prepared to run in Full mode... 11/07/2024 02:13:46.017 7f4d0bb66f48 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 11/07/2024 02:13:46.017 7f4d0bb66f48 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 11/07/2024 02:13:46.020 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page 11/07/2024 02:13:46.020 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 11/07/2024 02:13:46.021 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page 11/07/2024 02:13:46.021 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 11/07/2024 02:13:46.022 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 11/07/2024 02:13:46.022 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 11/07/2024 02:13:46.026 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page 11/07/2024 02:13:46.026 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 11/07/2024 02:13:46.026 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 11/07/2024 02:13:46.026 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 11/07/2024 02:13:46.026 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 11/07/2024 02:13:46.026 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 11/07/2024 02:13:46.026 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 11/07/2024 02:13:46.026 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 11/07/2024 02:13:46.026 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 11/07/2024 02:13:46.027 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 11/07/2024 02:13:46.027 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 11/07/2024 02:13:46.027 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 11/07/2024 02:13:46.027 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 11/07/2024 02:13:46.028 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 11/07/2024 02:13:46.028 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 11/07/2024 02:13:46.028 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 11/07/2024 02:13:46.028 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 11/07/2024 02:13:46.029 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 11/07/2024 02:13:46.029 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 11/07/2024 02:13:46.029 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 11/07/2024 02:13:46.029 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page 11/07/2024 02:13:46.030 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 11/07/2024 02:13:46.030 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 11/07/2024 02:13:46.030 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 11/07/2024 02:13:46.030 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 11/07/2024 02:13:46.030 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 11/07/2024 02:13:46.031 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page 11/07/2024 02:13:46.031 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 11/07/2024 02:13:46.031 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 11/07/2024 02:13:46.031 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 11/07/2024 02:13:46.031 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 11/07/2024 02:13:46.031 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 11/07/2024 02:13:46.031 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 11/07/2024 02:13:46.031 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 11/07/2024 02:13:46.031 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 11/07/2024 02:13:46.031 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 11/07/2024 02:13:46.032 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 11/07/2024 02:13:46.032 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 11/07/2024 02:13:46.032 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 11/07/2024 02:13:46.032 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 11/07/2024 02:13:46.032 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 11/07/2024 02:13:46.035 7f4d0bb66f48 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 11/07/2024 02:13:46.076 7f4d0267c630 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 11/07/2024 02:13:46.076 7f4d0267c630 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 11/07/2024 02:13:46.077 7f4d026d6e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:46.077 7f4d026d6e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[none to Idle on EvStart], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 11/07/2024 02:13:46.077 7f4d026d6e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:46.091 7f4d026ca198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:46.091 7f4d026ca198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:13:46.091 7f4d026ca198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:46.714 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:13:46.722 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:13:46.725 7f4d0b7a76d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:13:46.735 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..tftp_queue', 'device_manager_ztp.ztp_queue'] 11/07/2024 02:13:47.737 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:13:48.533 7f4d01ca8d68 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:13:48.534 7f4d01ca8d68 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:13:48.534 7f4d01ca8d68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:13:48.534 7f4d01ca8d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:13:48.534 7f4d01ca8d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:13:48.534 7f4d01ca8d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:13:48.535 7f4d01ca8d68 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:13:48.535 7f4d01ca8d68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:13:48.535 7f4d01ca8d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:48.535 7f4d01ca8d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:13:48.535 7f4d01ca8d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:48.535 7f4d01ca8d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:48.535 7f4d01ca8d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945624533267 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:13:48.535 7f4d01ca8d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:48.738 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:13:49.739 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:13:50.740 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:13:51.740 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:13:52.534 7f4d01ca8d68 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:13:52.535 7f4d01ca8d68 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:13:52.536 7f4d01ca8d68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:13:52.536 7f4d01ca8d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:13:52.536 7f4d01ca8d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:13:52.536 7f4d01ca8d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:13:52.536 7f4d01ca8d68 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:13:52.536 7f4d01ca8d68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:13:52.536 7f4d01ca8d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:52.537 7f4d01ca8d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:13:52.537 7f4d01ca8d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:52.537 7f4d01ca8d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:52.537 7f4d01ca8d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945624533267 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:13:52.537 7f4d01ca8d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:52.742 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:13:53.743 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:13:54.744 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:13:55.745 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:13:56.536 7f4d03d8a588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:13:56.537 7f4d03d8a588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:13:56.537 7f4d03d8a588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:13:56.538 7f4d03d8a588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:13:56.538 7f4d03d8a588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:13:56.538 7f4d03d8a588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:13:56.538 7f4d03d8a588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:13:56.538 7f4d03d8a588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:13:56.538 7f4d03d8a588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:56.538 7f4d03d8a588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:13:56.538 7f4d03d8a588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:56.538 7f4d03d8a588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:56.539 7f4d03d8a588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945624533267 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:13:56.539 7f4d03d8a588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:13:56.746 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:13:57.748 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:13:58.749 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:13:59.749 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:00.538 7f4d01ca8eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:14:00.540 7f4d01ca8eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:14:00.540 7f4d01ca8eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:14:00.540 7f4d01ca8eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:14:00.540 7f4d01ca8eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:14:00.540 7f4d01ca8eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:14:00.540 7f4d01ca8eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:14:00.541 7f4d01ca8eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:14:00.541 7f4d01ca8eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:00.541 7f4d01ca8eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:14:00.541 7f4d01ca8eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:00.541 7f4d01ca8eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:00.541 7f4d01ca8eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945624533267 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:14:00.541 7f4d01ca8eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:00.750 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:01.751 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:02.751 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:03.752 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:04.542 7f4d08ca0438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:14:04.543 7f4d08ca0438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:14:04.543 7f4d08ca0438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:14:04.543 7f4d08ca0438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:14:04.543 7f4d08ca0438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:14:04.543 7f4d08ca0438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:14:04.543 7f4d08ca0438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:14:04.543 7f4d08ca0438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:14:04.543 7f4d08ca0438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:04.544 7f4d08ca0438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:14:04.544 7f4d08ca0438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:04.544 7f4d08ca0438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:04.544 7f4d08ca0438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945624533267 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:14:04.544 7f4d08ca0438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:04.753 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:05.754 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:06.755 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:07.756 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:08.543 7f4d01cb4978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:14:08.546 7f4d01cb4978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:14:08.546 7f4d01cb4978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:14:08.547 7f4d01cb4978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:14:08.547 7f4d01cb4978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:14:08.547 7f4d01cb4978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:14:08.547 7f4d01cb4978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:14:08.547 7f4d01cb4978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:14:08.548 7f4d01cb4978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:08.548 7f4d01cb4978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:14:08.548 7f4d01cb4978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:08.548 7f4d01cb4978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:08.548 7f4d01cb4978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945624533267 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:14:08.548 7f4d01cb4978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:08.757 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:09.758 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:10.759 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:11.760 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:12.547 7f4d01cb4630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:14:12.548 7f4d01cb4630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:14:12.548 7f4d01cb4630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:14:12.548 7f4d01cb4630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:14:12.548 7f4d01cb4630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:14:12.548 7f4d01cb4630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:14:12.548 7f4d01cb4630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:14:12.548 7f4d01cb4630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:14:12.548 7f4d01cb4630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:12.549 7f4d01cb4630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:14:12.549 7f4d01cb4630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:12.549 7f4d01cb4630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:12.549 7f4d01cb4630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945624533267 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:14:12.549 7f4d01cb4630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:12.761 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:13.762 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:14.764 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:15.765 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:16.549 7f4d01cb4588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:14:16.550 7f4d01cb4588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:14:16.550 7f4d01cb4588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:14:16.550 7f4d01cb4588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:14:16.550 7f4d01cb4588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:14:16.550 7f4d01cb4588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:14:16.550 7f4d01cb4588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:14:16.551 7f4d01cb4588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:14:16.551 7f4d01cb4588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:16.551 7f4d01cb4588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:14:16.551 7f4d01cb4588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:16.551 7f4d01cb4588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:16.551 7f4d01cb4588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945624533267 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:14:16.551 7f4d01cb4588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:16.766 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:17.767 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:18.768 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:19.768 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:20.550 7f4d01cb4630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:14:20.553 7f4d01cb4630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:14:20.553 7f4d01cb4630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:14:20.553 7f4d01cb4630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:14:20.554 7f4d01cb4630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:14:20.554 7f4d01cb4630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:14:20.554 7f4d01cb4630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:14:20.554 7f4d01cb4630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:14:20.554 7f4d01cb4630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:20.554 7f4d01cb4630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:14:20.554 7f4d01cb4630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:20.554 7f4d01cb4630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:20.554 7f4d01cb4630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945624533267 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:14:20.555 7f4d01cb4630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:20.769 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:21.770 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:22.771 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:23.772 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:24.554 7f4d01cc4d68 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:14:24.555 7f4d01cc4d68 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:14:24.555 7f4d01cc4d68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:14:24.555 7f4d01cc4d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:14:24.555 7f4d01cc4d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:14:24.555 7f4d01cc4d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:14:24.555 7f4d01cc4d68 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:14:24.556 7f4d01cc4d68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:14:24.556 7f4d01cc4d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:24.556 7f4d01cc4d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:14:24.556 7f4d01cc4d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:24.556 7f4d01cc4d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:24.556 7f4d01cc4d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945624533267 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:14:24.556 7f4d01cc4d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:24.773 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:25.773 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:26.775 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:27.776 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:28.556 7f4d09df38d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:14:28.557 7f4d09df38d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:14:28.557 7f4d09df38d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:14:28.557 7f4d09df38d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:14:28.557 7f4d09df38d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:14:28.558 7f4d09df38d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:14:28.558 7f4d09df38d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:14:28.558 7f4d09df38d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:14:28.558 7f4d09df38d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:28.558 7f4d09df38d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:14:28.558 7f4d09df38d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:28.558 7f4d09df38d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:28.558 7f4d09df38d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945624533267 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:14:28.558 7f4d09df38d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:28.777 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:29.777 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:30.778 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:31.779 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:32.558 7f4d01ca8240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:14:32.559 7f4d01ca8240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:14:32.559 7f4d01ca8240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:14:32.559 7f4d01ca8240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:14:32.560 7f4d01ca8240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:14:32.560 7f4d01ca8240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:14:32.560 7f4d01ca8240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:14:32.560 7f4d01ca8240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:14:32.560 7f4d01ca8240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:32.560 7f4d01ca8240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:14:32.560 7f4d01ca8240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:32.560 7f4d01ca8240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:32.561 7f4d01ca8240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945624533267 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:14:32.561 7f4d01ca8240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:32.780 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:33.781 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:34.782 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:35.783 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:36.560 7f4d01cb4588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:14:36.561 7f4d01cb4588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:14:36.561 7f4d01cb4588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:14:36.561 7f4d01cb4588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:14:36.561 7f4d01cb4588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:14:36.561 7f4d01cb4588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:14:36.562 7f4d01cb4588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:14:36.562 7f4d01cb4588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:14:36.562 7f4d01cb4588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:36.562 7f4d01cb4588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:14:36.562 7f4d01cb4588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:36.562 7f4d01cb4588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:36.562 7f4d01cb4588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945624533267 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:14:36.562 7f4d01cb4588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:36.784 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:37.785 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:38.786 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:39.786 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:40.562 7f4d02728588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:14:40.563 7f4d02728588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:14:40.563 7f4d02728588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:14:40.563 7f4d02728588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:14:40.563 7f4d02728588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:14:40.564 7f4d02728588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:14:40.564 7f4d02728588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:14:40.564 7f4d02728588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:14:40.564 7f4d02728588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:40.564 7f4d02728588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:14:40.564 7f4d02728588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:40.564 7f4d02728588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:40.565 7f4d02728588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945624533267 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:14:40.565 7f4d02728588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:40.787 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:41.788 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:42.789 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:43.791 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:44.563 7f4d02728630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:14:44.564 7f4d02728630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 11/07/2024 02:14:44.565 7f4d02728630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:14:44.565 7f4d02728630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:14:44.565 7f4d02728630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:14:44.565 7f4d02728630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:14:44.565 7f4d02728630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/07/2024 02:14:44.565 7f4d02728630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/07/2024 02:14:44.566 7f4d02728630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:44.566 7f4d02728630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/07/2024 02:14:44.566 7f4d02728630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:44.566 7f4d02728630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:44.566 7f4d02728630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 114 http_port = 8096 start_time = 1730945624533267 collector_name = collector_ip = 10.0.0.35:8086 collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/07/2024 02:14:44.566 7f4d02728630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:14:44.792 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:14:45.336 7f4d0bb66f48 [contrail-device-manager] [ERROR]: network IPAM 'ipv6_link_local' already exists or other conflict: ['default-domain', 'default-project', '_internal_ipam_ipv6_link_local'] already exists with uuid: c7156525-7415-4bc0-8fb9-e096f9dc20b5 11/07/2024 02:14:45.488 7f4d0bb66f48 [contrail-device-manager] [ERROR]: virtual network '_internal_vn_ipv6_link_local' already exists or other conflict: ['default-domain', 'default-project', '_internal_vn_ipv6_link_local'] already exists with uuid: 586c4e60-0a4a-42d0-9d86-197b2f9aa7c5 11/07/2024 02:14:45.792 7f4d0b7a76d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:30.509 7f35b6293dc8 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 11/07/2024 02:15:30.510 7f35b6293dc8 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 11/07/2024 02:15:30.510 7f35b6293dc8 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 11/07/2024 02:15:30.510 7f35ad92b6d8 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 11/07/2024 02:15:30.510 7f35ad92b6d8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 11/07/2024 02:15:30.511 7f35ad92b6d8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/07/2024 02:15:30.511 7f35ad92b6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 11/07/2024 02:15:30.511 7f35ad92b6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:30.511 7f35ad92b6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:15:30.511 7f35ad92b6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:30.511 7f35ad92b6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:30.511 7f35ad92b6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 113 start_time = 1730945730510946 collector_name = collector_ip = collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 11/07/2024 02:15:30.511 7f35ad92b6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:30.547 7f35b6293dc8 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.35:5673//'] 11/07/2024 02:15:30.552 7f35b6293dc8 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 11/07/2024 02:15:30.552 7f35ad87b240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:30.552 7f35ad87b240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/07/2024 02:15:30.552 7f35ad87b240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:30.552 7f35ad87b240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:30.553 7f35ad87b240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/07/2024 02:15:30.553 7f35ad87b240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:30.559 7f35b6293dc8 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 11/07/2024 02:15:30.560 7f35ad87b0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:30.560 7f35ad87b0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:15:30.560 7f35ad87b0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:30.567 7f35b6293dc8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 11/07/2024 02:15:30.567 7f35ad7d42e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:30.567 7f35ad7d42e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/07/2024 02:15:30.568 7f35ad7d42e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:30.570 7f35b6293dc8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 11/07/2024 02:15:30.570 7f35ad7d42e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:30.570 7f35ad7d42e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:15:30.570 7f35ad7d42e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:30.580 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 11/07/2024 02:15:30.580 7f35b6293dc8 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 11/07/2024 02:15:30.580 7f35b6293dc8 [contrail-device-manager] [WARNING]: initializing cassandra cluster 11/07/2024 02:15:30.581 7f35b6293dc8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 11/07/2024 02:15:30.581 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 11/07/2024 02:15:30.581 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:15:30.581 7f35b033e0f0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 11/07/2024 02:15:30.582 7f35b033e0f0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 11/07/2024 02:15:30.582 7f35ad7d4cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:30.582 7f35ad7d4cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/07/2024 02:15:30.582 7f35ad7d4cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:30.583 7f35b6293dc8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 11/07/2024 02:15:30.588 7f35b033e0f0 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 11/07/2024 02:15:30.590 7f35b5ee5630 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:15:30.590 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 11/07/2024 02:15:30.612 7f35b6293dc8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 11/07/2024 02:15:30.615 7f35ace748d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:30.615 7f35ace748d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 11/07/2024 02:15:30.615 7f35ace748d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:30.624 7f35b6293dc8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 11/07/2024 02:15:30.624 7f35acf19828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:30.625 7f35acf19828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/07/2024 02:15:30.625 7f35acf19828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:30.629 7f35b6293dc8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 11/07/2024 02:15:30.639 7f35b6293dc8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 11/07/2024 02:15:30.640 7f35aceab2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:30.640 7f35aceab2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:15:30.640 7f35aceab2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:30.653 7f35b06378d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:30.653 7f35b06378d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:Zookeeper[] connection down >>, ] >> 11/07/2024 02:15:30.653 7f35b06378d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:30.659 7f35aceae048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:30.659 7f35aceae048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:15:30.660 7f35aceae048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:30.660 7f35b6293dc8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 11/07/2024 02:15:30.660 7f35b6293dc8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 11/07/2024 02:15:30.660 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:15:30.660 7f35b6293dc8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1. 11/07/2024 02:15:30.669 7f35b5ee5630 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:15:30.676 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.'] 11/07/2024 02:15:30.685 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:15:30.689 7f35b5ee5630 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:15:30.696 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.'] 11/07/2024 02:15:31.172 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Fetching all containers 11/07/2024 02:15:31.406 7f35b6293dc8 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 11/07/2024 02:15:31.589 7f35b6293dc8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..config_queue 11/07/2024 02:15:31.589 7f35b6293dc8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..tftp_queue 11/07/2024 02:15:31.698 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:31.706 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:15:31.720 7f35b5ee5630 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:15:31.731 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..tftp_queue'] 11/07/2024 02:15:32.090 7f35b6293dc8 [contrail-device-manager] [WARNING]: Process 113 prepared to run in Full mode... 11/07/2024 02:15:32.090 7f35b6293dc8 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 11/07/2024 02:15:32.090 7f35b6293dc8 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 11/07/2024 02:15:32.095 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page 11/07/2024 02:15:32.095 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 11/07/2024 02:15:32.096 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page 11/07/2024 02:15:32.097 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 11/07/2024 02:15:32.098 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 11/07/2024 02:15:32.098 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 11/07/2024 02:15:32.104 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page 11/07/2024 02:15:32.104 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 11/07/2024 02:15:32.105 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 11/07/2024 02:15:32.105 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 11/07/2024 02:15:32.105 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 11/07/2024 02:15:32.105 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 11/07/2024 02:15:32.105 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 11/07/2024 02:15:32.105 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 11/07/2024 02:15:32.105 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 11/07/2024 02:15:32.106 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 11/07/2024 02:15:32.106 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 11/07/2024 02:15:32.106 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 11/07/2024 02:15:32.106 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 11/07/2024 02:15:32.108 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 11/07/2024 02:15:32.108 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 11/07/2024 02:15:32.108 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 11/07/2024 02:15:32.109 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 11/07/2024 02:15:32.109 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 11/07/2024 02:15:32.109 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 11/07/2024 02:15:32.110 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 11/07/2024 02:15:32.110 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page 11/07/2024 02:15:32.110 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 11/07/2024 02:15:32.111 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 11/07/2024 02:15:32.111 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 11/07/2024 02:15:32.111 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 11/07/2024 02:15:32.111 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 11/07/2024 02:15:32.112 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page 11/07/2024 02:15:32.112 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 11/07/2024 02:15:32.112 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 11/07/2024 02:15:32.112 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 11/07/2024 02:15:32.113 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 11/07/2024 02:15:32.113 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 11/07/2024 02:15:32.113 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 11/07/2024 02:15:32.113 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 11/07/2024 02:15:32.113 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 11/07/2024 02:15:32.113 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 11/07/2024 02:15:32.113 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 11/07/2024 02:15:32.113 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 11/07/2024 02:15:32.114 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 11/07/2024 02:15:32.114 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 11/07/2024 02:15:32.114 7f35b6293dc8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 11/07/2024 02:15:32.117 7f35b6293dc8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 11/07/2024 02:15:32.158 7f35acd6c4e0 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 11/07/2024 02:15:32.159 7f35acd6c4e0 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 11/07/2024 02:15:32.159 7f35acdb3e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:32.159 7f35acdb3e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[none to Idle on EvStart], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 11/07/2024 02:15:32.159 7f35acdb3e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:32.173 7f35acbf2978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:32.173 7f35acbf2978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:15:32.174 7f35acbf2978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:15:32.736 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:32.750 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:15:32.754 7f35b5ee5630 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:15:32.766 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..tftp_queue', 'device_manager_ztp.ztp_queue'] 11/07/2024 02:15:33.770 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:34.511 7f35ac3f8978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:15:34.512 7f35ac3f8978 [contrail-device-manager] [INFO]: Session Event: TCP Connected 11/07/2024 02:15:34.513 7f35ac3f8978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:15:34.513 7f35ac3f8978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:15:34.513 7f35ac3f8978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:15:34.513 7f35ac3f8978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:15:34.513 7f35ac3f8978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect] 11/07/2024 02:15:34.513 7f35ac3f8978 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19 11/07/2024 02:15:34.513 7f35ac3f8978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 11/07/2024 02:15:34.514 7f35ac3f8978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/07/2024 02:15:34.514 7f35ac3f8978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/07/2024 02:15:34.514 7f35ac406438 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 11/07/2024 02:15:34.515 7f35ac406438 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 11/07/2024 02:15:34.518 7f35ac4062e8 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001017 11/07/2024 02:15:34.518 7f35ac4062e8 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 11/07/2024 02:15:34.519 7f35ae4d9eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 11/07/2024 02:15:34.519 7f35ae4d9eb8 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 0 11/07/2024 02:15:34.519 7f35ae4d9eb8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 11/07/2024 02:15:34.519 7f35ae4d9eb8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/07/2024 02:15:34.519 7f35ae4d9eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 11/07/2024 02:15:34.520 7f35ac406048 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 11/07/2024 02:15:34.521 7f35ac406048 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 11/07/2024 02:15:34.771 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:35.772 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:36.773 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:37.774 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:38.775 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:39.776 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:40.777 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:41.778 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:42.779 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:43.779 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:44.782 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:45.783 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:46.783 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:47.785 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:48.785 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:49.788 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:50.789 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:51.790 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:52.791 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:53.792 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:54.793 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:55.794 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:56.795 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:57.796 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:58.797 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:15:59.798 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:00.799 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:01.799 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:02.800 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:03.801 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:04.802 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:05.803 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:06.805 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:07.805 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:08.807 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:09.808 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:10.809 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:11.809 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:12.811 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:13.811 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:14.812 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:15.813 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:16.814 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:17.815 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:18.816 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:19.818 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:20.819 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:21.820 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:22.822 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:23.822 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:24.823 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:25.825 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:26.825 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:27.826 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:28.827 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:29.828 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:30.829 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:16:31.284 7f35b6293dc8 [contrail-device-manager] [ERROR]: network IPAM 'ipv6_link_local' already exists or other conflict: ['default-domain', 'default-project', '_internal_ipam_ipv6_link_local'] already exists with uuid: c7156525-7415-4bc0-8fb9-e096f9dc20b5 11/07/2024 02:16:31.436 7f35b6293dc8 [contrail-device-manager] [ERROR]: virtual network '_internal_vn_ipv6_link_local' already exists or other conflict: ['default-domain', 'default-project', '_internal_vn_ipv6_link_local'] already exists with uuid: 586c4e60-0a4a-42d0-9d86-197b2f9aa7c5 11/07/2024 02:16:31.888 7f35b5ee5630 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:16.997 7f9a549b1d08 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 11/07/2024 02:17:16.998 7f9a549b1d08 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 11/07/2024 02:17:16.998 7f9a549b1d08 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 11/07/2024 02:17:16.998 7f9a4c01c6d8 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 11/07/2024 02:17:16.998 7f9a4c01c6d8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 11/07/2024 02:17:16.999 7f9a4c01c6d8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/07/2024 02:17:16.999 7f9a4c01c6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 11/07/2024 02:17:16.999 7f9a4c01c6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:16.999 7f9a4c01c6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:17:16.999 7f9a4c01c6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:16.999 7f9a4c01c6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:16.999 7f9a4c01c6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 113 start_time = 1730945836999230 collector_name = collector_ip = collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 11/07/2024 02:17:17.000 7f9a4c01c6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:17.047 7f9a549b1d08 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.35:5673//'] 11/07/2024 02:17:17.055 7f9a549b1d08 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 11/07/2024 02:17:17.055 7f9a47f57240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:17.056 7f9a47f57240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/07/2024 02:17:17.056 7f9a47f57240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:17.056 7f9a47f57240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:17.056 7f9a47f57240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/07/2024 02:17:17.056 7f9a47f57240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:17.059 7f9a549b1d08 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 11/07/2024 02:17:17.060 7f9a47f570f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:17.060 7f9a47f570f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:17:17.060 7f9a47f570f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:17.068 7f9a549b1d08 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 11/07/2024 02:17:17.068 7f9a47eb02e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:17.068 7f9a47eb02e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/07/2024 02:17:17.068 7f9a47eb02e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:17.071 7f9a549b1d08 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 11/07/2024 02:17:17.071 7f9a47eb02e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:17.071 7f9a47eb02e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:17:17.071 7f9a47eb02e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:17.077 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 11/07/2024 02:17:17.077 7f9a549b1d08 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 11/07/2024 02:17:17.078 7f9a549b1d08 [contrail-device-manager] [WARNING]: initializing cassandra cluster 11/07/2024 02:17:17.078 7f9a549b1d08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 11/07/2024 02:17:17.078 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 11/07/2024 02:17:17.078 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:17:17.079 7f9a4ea2f0f0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 11/07/2024 02:17:17.079 7f9a4ea2f0f0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 11/07/2024 02:17:17.079 7f9a47eb0cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:17.079 7f9a47eb0cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/07/2024 02:17:17.079 7f9a47eb0cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:17.080 7f9a549b1d08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 11/07/2024 02:17:17.085 7f9a4ea2f0f0 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 11/07/2024 02:17:17.086 7f9a54861c18 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:17:17.086 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 11/07/2024 02:17:17.118 7f9a549b1d08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 11/07/2024 02:17:17.120 7f9a475528d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:17.120 7f9a475528d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 11/07/2024 02:17:17.120 7f9a475528d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:17.126 7f9a549b1d08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 11/07/2024 02:17:17.126 7f9a47554828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:17.126 7f9a47554828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/07/2024 02:17:17.126 7f9a47554828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:17.131 7f9a549b1d08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 11/07/2024 02:17:17.142 7f9a549b1d08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 11/07/2024 02:17:17.143 7f9a475662e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:17.143 7f9a475662e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:17:17.143 7f9a475662e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:17.159 7f9a47e74438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:17.160 7f9a47e74438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:Zookeeper[] connection down >>, ] >> 11/07/2024 02:17:17.160 7f9a47e74438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:17.166 7f9a47563eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:17.167 7f9a47563eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:17:17.167 7f9a47563eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:17.167 7f9a549b1d08 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 11/07/2024 02:17:17.167 7f9a549b1d08 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 11/07/2024 02:17:17.168 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:17:17.168 7f9a549b1d08 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1. 11/07/2024 02:17:17.174 7f9a54861c18 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:17:17.180 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.'] 11/07/2024 02:17:17.195 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:17:17.205 7f9a54861c18 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:17:17.210 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.'] 11/07/2024 02:17:17.685 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Fetching all containers 11/07/2024 02:17:17.949 7f9a549b1d08 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 11/07/2024 02:17:18.212 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:18.294 7f9a549b1d08 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..config_queue 11/07/2024 02:17:18.294 7f9a549b1d08 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..tftp_queue 11/07/2024 02:17:18.795 7f9a549b1d08 [contrail-device-manager] [WARNING]: Process 113 prepared to run in Full mode... 11/07/2024 02:17:18.795 7f9a549b1d08 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 11/07/2024 02:17:18.795 7f9a549b1d08 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 11/07/2024 02:17:18.801 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page 11/07/2024 02:17:18.801 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 11/07/2024 02:17:18.803 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page 11/07/2024 02:17:18.803 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 11/07/2024 02:17:18.805 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 11/07/2024 02:17:18.805 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 11/07/2024 02:17:18.813 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page 11/07/2024 02:17:18.813 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 11/07/2024 02:17:18.813 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 11/07/2024 02:17:18.814 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 11/07/2024 02:17:18.814 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 11/07/2024 02:17:18.814 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 11/07/2024 02:17:18.814 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 11/07/2024 02:17:18.814 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 11/07/2024 02:17:18.814 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 11/07/2024 02:17:18.814 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 11/07/2024 02:17:18.814 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 11/07/2024 02:17:18.814 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 11/07/2024 02:17:18.814 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 11/07/2024 02:17:18.816 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 11/07/2024 02:17:18.816 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 11/07/2024 02:17:18.816 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 11/07/2024 02:17:18.817 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 11/07/2024 02:17:18.817 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 11/07/2024 02:17:18.817 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 11/07/2024 02:17:18.817 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 11/07/2024 02:17:18.818 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page 11/07/2024 02:17:18.818 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 11/07/2024 02:17:18.818 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 11/07/2024 02:17:18.818 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 11/07/2024 02:17:18.818 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 11/07/2024 02:17:18.819 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 11/07/2024 02:17:18.819 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page 11/07/2024 02:17:18.819 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 11/07/2024 02:17:18.820 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 11/07/2024 02:17:18.820 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 11/07/2024 02:17:18.820 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 11/07/2024 02:17:18.820 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 11/07/2024 02:17:18.820 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 11/07/2024 02:17:18.820 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 11/07/2024 02:17:18.820 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 11/07/2024 02:17:18.820 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 11/07/2024 02:17:18.820 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 11/07/2024 02:17:18.820 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 11/07/2024 02:17:18.820 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 11/07/2024 02:17:18.820 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 11/07/2024 02:17:18.820 7f9a549b1d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 11/07/2024 02:17:18.823 7f9a549b1d08 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 11/07/2024 02:17:18.912 7f9a47291e10 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 11/07/2024 02:17:18.912 7f9a47291e10 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 11/07/2024 02:17:18.913 7f9a4738fac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:18.913 7f9a4738fac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[none to Idle on EvStart], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 11/07/2024 02:17:18.913 7f9a4738fac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:18.942 7f9a4739a2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:18.942 7f9a4739a2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:17:18.942 7f9a4739a2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:17:19.212 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:19.222 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:17:19.235 7f9a54861c18 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:17:19.252 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..tftp_queue', 'device_manager_ztp.ztp_queue'] 11/07/2024 02:17:20.255 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:20.999 7f9a4739a2e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:17:21.000 7f9a4739a2e8 [contrail-device-manager] [INFO]: Session Event: TCP Connected 11/07/2024 02:17:21.000 7f9a4739a2e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:17:21.000 7f9a4739a2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:17:21.000 7f9a4739a2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:17:21.000 7f9a4739a2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:17:21.000 7f9a4739a2e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect] 11/07/2024 02:17:21.001 7f9a4739a2e8 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19 11/07/2024 02:17:21.001 7f9a4739a2e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 11/07/2024 02:17:21.001 7f9a4739a2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/07/2024 02:17:21.001 7f9a4739a2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/07/2024 02:17:21.001 7f9a46acda20 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 11/07/2024 02:17:21.002 7f9a46acda20 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 11/07/2024 02:17:21.005 7f9a46ad78d0 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001017 11/07/2024 02:17:21.005 7f9a46ad78d0 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 11/07/2024 02:17:21.005 7f9a4cbcbeb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 11/07/2024 02:17:21.006 7f9a4cbcbeb8 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 0 11/07/2024 02:17:21.006 7f9a4cbcbeb8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 11/07/2024 02:17:21.006 7f9a4cbcbeb8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/07/2024 02:17:21.006 7f9a4cbcbeb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 11/07/2024 02:17:21.007 7f9a4cc27630 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 11/07/2024 02:17:21.009 7f9a4cc27630 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 11/07/2024 02:17:21.255 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:22.256 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:23.257 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:24.257 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:25.258 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:26.259 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:27.260 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:28.260 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:29.261 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:30.262 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:31.265 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:32.266 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:33.267 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:34.267 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:35.269 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:36.270 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:37.271 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:38.485 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:39.486 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:40.487 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:41.488 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:42.489 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:43.489 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:44.491 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:45.491 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:46.492 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:47.493 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:48.493 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:49.495 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:50.495 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:51.496 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:52.497 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:53.498 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:54.498 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:55.500 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:56.500 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:57.501 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:58.502 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:59.502 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:17:59.942 7f9a46b858d0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 11/07/2024 02:18:00.503 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:18:01.504 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:18:02.505 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:18:03.506 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:18:04.508 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:18:05.508 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:18:06.510 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:18:07.511 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:18:08.512 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:18:09.513 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:18:10.515 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:18:11.515 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:18:12.516 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:18:13.517 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:18:14.519 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:18:15.520 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:18:16.521 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:18:17.522 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:18:18.144 7f9a549b1d08 [contrail-device-manager] [ERROR]: network IPAM 'ipv6_link_local' already exists or other conflict: ['default-domain', 'default-project', '_internal_ipam_ipv6_link_local'] already exists with uuid: c7156525-7415-4bc0-8fb9-e096f9dc20b5 11/07/2024 02:18:18.300 7f9a549b1d08 [contrail-device-manager] [ERROR]: virtual network '_internal_vn_ipv6_link_local' already exists or other conflict: ['default-domain', 'default-project', '_internal_vn_ipv6_link_local'] already exists with uuid: 586c4e60-0a4a-42d0-9d86-197b2f9aa7c5 11/07/2024 02:18:18.523 7f9a54861c18 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:02.545 7f532a465048 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 11/07/2024 02:19:02.545 7f532a465048 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 11/07/2024 02:19:02.545 7f532a465048 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 11/07/2024 02:19:02.545 7f5321b046d8 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 11/07/2024 02:19:02.545 7f5321b046d8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 11/07/2024 02:19:02.546 7f5321b046d8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/07/2024 02:19:02.546 7f5321b046d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 11/07/2024 02:19:02.546 7f5321b046d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:02.546 7f5321b046d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:19:02.546 7f5321b046d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:02.546 7f5321b046d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:02.546 7f5321b046d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 113 start_time = 1730945942546049 collector_name = collector_ip = collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 11/07/2024 02:19:02.546 7f5321b046d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:02.582 7f532a465048 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.35:5673//'] 11/07/2024 02:19:02.587 7f532a465048 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 11/07/2024 02:19:02.587 7f5321a54240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:02.587 7f5321a54240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/07/2024 02:19:02.587 7f5321a54240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:02.588 7f5321a54240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:02.588 7f5321a54240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/07/2024 02:19:02.588 7f5321a54240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:02.590 7f532a465048 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 11/07/2024 02:19:02.591 7f5321a540f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:02.591 7f5321a540f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:19:02.591 7f5321a540f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:02.597 7f532a465048 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 11/07/2024 02:19:02.597 7f53219ad2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:02.598 7f53219ad2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/07/2024 02:19:02.598 7f53219ad2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:02.600 7f532a465048 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 11/07/2024 02:19:02.600 7f53219ad2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:02.601 7f53219ad2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:19:02.601 7f53219ad2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:02.608 7f532a465048 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 11/07/2024 02:19:02.609 7f532a465048 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 11/07/2024 02:19:02.609 7f532a465048 [contrail-device-manager] [WARNING]: initializing cassandra cluster 11/07/2024 02:19:02.609 7f532a465048 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 11/07/2024 02:19:02.610 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 11/07/2024 02:19:02.610 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:19:02.610 7f53245160f0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 11/07/2024 02:19:02.610 7f53245160f0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 11/07/2024 02:19:02.610 7f53219adcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:02.610 7f53219adcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/07/2024 02:19:02.610 7f53219adcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:02.612 7f532a465048 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 11/07/2024 02:19:02.617 7f532a340b70 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:19:02.617 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 11/07/2024 02:19:02.617 7f53245160f0 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 11/07/2024 02:19:02.636 7f532a465048 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 11/07/2024 02:19:02.637 7f532184f780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:02.637 7f532184f780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 11/07/2024 02:19:02.638 7f532184f780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:02.642 7f532a465048 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 11/07/2024 02:19:02.642 7f5321854c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:02.642 7f5321854c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/07/2024 02:19:02.642 7f5321854c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:02.645 7f532a465048 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 11/07/2024 02:19:02.653 7f532a465048 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 11/07/2024 02:19:02.653 7f53218666d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:02.653 7f53218666d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:19:02.653 7f53218666d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:02.669 7f532195bf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:02.669 7f532195bf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:Zookeeper[] connection down >>, ] >> 11/07/2024 02:19:02.669 7f532195bf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:02.677 7f532185f4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:02.677 7f532185f4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:19:02.677 7f532185f4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:02.677 7f532a465048 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 11/07/2024 02:19:02.677 7f532a465048 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 11/07/2024 02:19:02.677 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:19:02.678 7f532a465048 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1. 11/07/2024 02:19:02.687 7f532a340b70 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:19:02.694 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.'] 11/07/2024 02:19:02.701 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:19:02.704 7f532a340b70 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:19:02.711 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.'] 11/07/2024 02:19:03.189 7f532a465048 [contrail-device-manager] [DEBUG]: Fetching all containers 11/07/2024 02:19:03.445 7f532a465048 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 11/07/2024 02:19:03.609 7f532a465048 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..config_queue 11/07/2024 02:19:03.609 7f532a465048 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..tftp_queue 11/07/2024 02:19:03.713 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:03.719 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:19:03.723 7f532a340b70 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:19:03.731 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..tftp_queue'] 11/07/2024 02:19:04.111 7f532a465048 [contrail-device-manager] [WARNING]: Process 113 prepared to run in Full mode... 11/07/2024 02:19:04.111 7f532a465048 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 11/07/2024 02:19:04.111 7f532a465048 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 11/07/2024 02:19:04.115 7f532a465048 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page 11/07/2024 02:19:04.115 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 11/07/2024 02:19:04.116 7f532a465048 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page 11/07/2024 02:19:04.117 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 11/07/2024 02:19:04.119 7f532a465048 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 11/07/2024 02:19:04.119 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 11/07/2024 02:19:04.124 7f532a465048 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page 11/07/2024 02:19:04.124 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 11/07/2024 02:19:04.124 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 11/07/2024 02:19:04.124 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 11/07/2024 02:19:04.125 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 11/07/2024 02:19:04.125 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 11/07/2024 02:19:04.125 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 11/07/2024 02:19:04.125 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 11/07/2024 02:19:04.125 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 11/07/2024 02:19:04.125 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 11/07/2024 02:19:04.125 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 11/07/2024 02:19:04.125 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 11/07/2024 02:19:04.125 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 11/07/2024 02:19:04.127 7f532a465048 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 11/07/2024 02:19:04.127 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 11/07/2024 02:19:04.127 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 11/07/2024 02:19:04.127 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 11/07/2024 02:19:04.128 7f532a465048 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 11/07/2024 02:19:04.128 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 11/07/2024 02:19:04.128 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 11/07/2024 02:19:04.129 7f532a465048 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page 11/07/2024 02:19:04.129 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 11/07/2024 02:19:04.129 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 11/07/2024 02:19:04.129 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 11/07/2024 02:19:04.129 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 11/07/2024 02:19:04.130 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 11/07/2024 02:19:04.131 7f532a465048 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page 11/07/2024 02:19:04.131 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 11/07/2024 02:19:04.131 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 11/07/2024 02:19:04.131 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 11/07/2024 02:19:04.131 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 11/07/2024 02:19:04.131 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 11/07/2024 02:19:04.131 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 11/07/2024 02:19:04.131 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 11/07/2024 02:19:04.131 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 11/07/2024 02:19:04.132 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 11/07/2024 02:19:04.132 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 11/07/2024 02:19:04.132 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 11/07/2024 02:19:04.132 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 11/07/2024 02:19:04.132 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 11/07/2024 02:19:04.132 7f532a465048 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 11/07/2024 02:19:04.135 7f532a465048 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 11/07/2024 02:19:04.190 7f5321661e10 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 11/07/2024 02:19:04.190 7f5321661e10 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 11/07/2024 02:19:04.190 7f532163a588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:04.190 7f532163a588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[none to Idle on EvStart], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 11/07/2024 02:19:04.190 7f532163a588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:04.206 7f5321697390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:04.206 7f5321697390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:19:04.206 7f5321697390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:19:04.732 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:04.745 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:19:04.753 7f532a340b70 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:19:04.775 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..tftp_queue', 'device_manager_ztp.ztp_queue'] 11/07/2024 02:19:05.780 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:06.546 7f5320dcfeb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:19:06.547 7f5320dcfeb8 [contrail-device-manager] [INFO]: Session Event: TCP Connected 11/07/2024 02:19:06.547 7f5320dcfeb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:19:06.547 7f5320dcfeb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:19:06.547 7f5320dcfeb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:19:06.547 7f5320dcfeb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:19:06.547 7f5320dcfeb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect] 11/07/2024 02:19:06.548 7f5320dcfeb8 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19 11/07/2024 02:19:06.548 7f5320dcfeb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 11/07/2024 02:19:06.548 7f5320dcfeb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/07/2024 02:19:06.548 7f5320dcfeb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/07/2024 02:19:06.548 7f5320de8438 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 11/07/2024 02:19:06.549 7f5320de8438 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 11/07/2024 02:19:06.553 7f5320de82e8 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001017 11/07/2024 02:19:06.553 7f5320de82e8 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 11/07/2024 02:19:06.553 7f53226b3eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 11/07/2024 02:19:06.554 7f53226b3eb8 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 0 11/07/2024 02:19:06.554 7f53226b3eb8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 11/07/2024 02:19:06.554 7f53226b3eb8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/07/2024 02:19:06.554 7f53226b3eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 11/07/2024 02:19:06.554 7f5320de8048 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 11/07/2024 02:19:06.555 7f5320de8048 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 11/07/2024 02:19:06.780 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:07.781 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:08.782 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:09.783 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:10.784 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:11.784 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:12.788 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:13.789 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:14.791 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:15.792 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:16.793 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:17.794 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:18.795 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:19.797 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:20.797 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:21.799 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:22.800 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:23.801 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:24.802 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:25.803 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:26.804 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:27.805 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:28.806 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:29.807 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:30.808 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:31.809 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:32.809 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:33.810 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:34.812 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:35.813 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:36.813 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:37.815 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:38.816 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:39.817 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:40.818 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:41.820 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:42.820 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:43.823 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:44.824 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:45.826 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:46.827 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:47.828 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:48.829 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:49.830 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:50.831 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:51.832 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:52.833 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:53.835 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:54.835 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:55.837 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:56.838 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:57.839 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:58.840 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:19:59.841 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:20:00.842 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:20:01.843 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:20:02.844 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:20:03.845 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:20:04.847 7f532a340b70 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:20:05.032 7f532a465048 [contrail-device-manager] [ERROR]: network IPAM 'ipv6_link_local' already exists or other conflict: ['default-domain', 'default-project', '_internal_ipam_ipv6_link_local'] already exists with uuid: c7156525-7415-4bc0-8fb9-e096f9dc20b5 11/07/2024 02:20:05.192 7f532a465048 [contrail-device-manager] [ERROR]: virtual network '_internal_vn_ipv6_link_local' already exists or other conflict: ['default-domain', 'default-project', '_internal_vn_ipv6_link_local'] already exists with uuid: 586c4e60-0a4a-42d0-9d86-197b2f9aa7c5 11/07/2024 02:20:48.588 7f903ff8ff88 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 11/07/2024 02:20:48.588 7f903ff8ff88 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 11/07/2024 02:20:48.588 7f903ff8ff88 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 11/07/2024 02:20:48.588 7f90376036d8 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 11/07/2024 02:20:48.588 7f90376036d8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 11/07/2024 02:20:48.589 7f90376036d8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/07/2024 02:20:48.589 7f90376036d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 11/07/2024 02:20:48.589 7f90376036d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:48.589 7f90376036d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:20:48.589 7f90376036d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:48.590 7f90376036d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:48.590 7f90376036d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 113 start_time = 1730946048589365 collector_name = collector_ip = collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 11/07/2024 02:20:48.590 7f90376036d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:48.626 7f903ff8ff88 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.35:5673//'] 11/07/2024 02:20:48.630 7f903ff8ff88 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 11/07/2024 02:20:48.630 7f9037551240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:48.631 7f9037551240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/07/2024 02:20:48.631 7f9037551240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:48.631 7f9037551240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:48.631 7f9037551240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/07/2024 02:20:48.631 7f9037551240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:48.634 7f903ff8ff88 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 11/07/2024 02:20:48.634 7f90375510f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:48.634 7f90375510f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:20:48.634 7f90375510f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:48.642 7f903ff8ff88 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 11/07/2024 02:20:48.642 7f90374aa2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:48.642 7f90374aa2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/07/2024 02:20:48.643 7f90374aa2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:48.645 7f903ff8ff88 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 11/07/2024 02:20:48.646 7f90374aa2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:48.646 7f90374aa2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:20:48.646 7f90374aa2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:48.652 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 11/07/2024 02:20:48.652 7f903ff8ff88 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 11/07/2024 02:20:48.652 7f903ff8ff88 [contrail-device-manager] [WARNING]: initializing cassandra cluster 11/07/2024 02:20:48.652 7f903ff8ff88 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 11/07/2024 02:20:48.652 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 11/07/2024 02:20:48.653 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:20:48.653 7f903a0160f0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 11/07/2024 02:20:48.653 7f903a0160f0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 11/07/2024 02:20:48.654 7f90374aacc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:48.654 7f90374aacc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/07/2024 02:20:48.654 7f90374aacc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:48.655 7f903ff8ff88 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 11/07/2024 02:20:48.660 7f903fbd0828 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:20:48.660 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 11/07/2024 02:20:48.660 7f903a0160f0 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 11/07/2024 02:20:48.680 7f903ff8ff88 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 11/07/2024 02:20:48.681 7f9036b4b8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:48.681 7f9036b4b8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 11/07/2024 02:20:48.681 7f9036b4b8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:48.685 7f903ff8ff88 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 11/07/2024 02:20:48.685 7f9036bed828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:48.686 7f9036bed828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/07/2024 02:20:48.686 7f9036bed828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:48.689 7f903ff8ff88 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 11/07/2024 02:20:48.697 7f903ff8ff88 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 11/07/2024 02:20:48.697 7f9036b5e2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:48.697 7f9036b5e2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:20:48.698 7f9036b5e2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:48.712 7f9037452c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:48.712 7f9037452c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:Zookeeper[] connection down >>, ] >> 11/07/2024 02:20:48.712 7f9037452c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:48.717 7f9036bed828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:48.718 7f9036bed828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:20:48.718 7f9036bed828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:48.718 7f903ff8ff88 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 11/07/2024 02:20:48.718 7f903ff8ff88 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 11/07/2024 02:20:48.719 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:20:48.719 7f903ff8ff88 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1. 11/07/2024 02:20:48.722 7f903fbd0828 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:20:48.729 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.'] 11/07/2024 02:20:48.735 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:20:48.739 7f903fbd0828 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:20:48.745 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.'] 11/07/2024 02:20:49.231 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Fetching all containers 11/07/2024 02:20:49.487 7f903ff8ff88 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 11/07/2024 02:20:49.697 7f903ff8ff88 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..config_queue 11/07/2024 02:20:49.697 7f903ff8ff88 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..tftp_queue 11/07/2024 02:20:49.747 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:20:49.760 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:20:49.771 7f903fbd0828 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:20:49.782 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..tftp_queue'] 11/07/2024 02:20:50.198 7f903ff8ff88 [contrail-device-manager] [WARNING]: Process 113 prepared to run in Full mode... 11/07/2024 02:20:50.198 7f903ff8ff88 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 11/07/2024 02:20:50.198 7f903ff8ff88 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 11/07/2024 02:20:50.201 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page 11/07/2024 02:20:50.201 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 11/07/2024 02:20:50.202 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page 11/07/2024 02:20:50.202 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 11/07/2024 02:20:50.203 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 11/07/2024 02:20:50.203 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 11/07/2024 02:20:50.208 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page 11/07/2024 02:20:50.208 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 11/07/2024 02:20:50.208 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 11/07/2024 02:20:50.208 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 11/07/2024 02:20:50.208 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 11/07/2024 02:20:50.208 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 11/07/2024 02:20:50.208 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 11/07/2024 02:20:50.208 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 11/07/2024 02:20:50.208 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 11/07/2024 02:20:50.208 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 11/07/2024 02:20:50.209 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 11/07/2024 02:20:50.209 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 11/07/2024 02:20:50.209 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 11/07/2024 02:20:50.210 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 11/07/2024 02:20:50.210 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 11/07/2024 02:20:50.210 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 11/07/2024 02:20:50.210 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 11/07/2024 02:20:50.211 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 11/07/2024 02:20:50.211 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 11/07/2024 02:20:50.211 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 11/07/2024 02:20:50.212 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page 11/07/2024 02:20:50.212 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 11/07/2024 02:20:50.212 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 11/07/2024 02:20:50.212 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 11/07/2024 02:20:50.212 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 11/07/2024 02:20:50.212 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 11/07/2024 02:20:50.213 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page 11/07/2024 02:20:50.213 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 11/07/2024 02:20:50.213 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 11/07/2024 02:20:50.213 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 11/07/2024 02:20:50.213 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 11/07/2024 02:20:50.213 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 11/07/2024 02:20:50.213 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 11/07/2024 02:20:50.214 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 11/07/2024 02:20:50.214 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 11/07/2024 02:20:50.214 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 11/07/2024 02:20:50.214 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 11/07/2024 02:20:50.214 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 11/07/2024 02:20:50.214 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 11/07/2024 02:20:50.214 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 11/07/2024 02:20:50.214 7f903ff8ff88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 11/07/2024 02:20:50.217 7f903ff8ff88 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 11/07/2024 02:20:50.258 7f9036a05d68 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 11/07/2024 02:20:50.258 7f9036a05d68 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 11/07/2024 02:20:50.258 7f90369a8198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:50.258 7f90369a8198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[none to Idle on EvStart], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 11/07/2024 02:20:50.258 7f90369a8198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:50.274 7f903681e4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:50.274 7f903681e4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:20:50.274 7f903681e4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:20:50.784 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:20:50.792 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:20:50.797 7f903fbd0828 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:20:50.815 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..tftp_queue', 'device_manager_ztp.ztp_queue'] 11/07/2024 02:20:51.817 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:20:52.589 7f90360d0e10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:20:52.590 7f90360d0e10 [contrail-device-manager] [INFO]: Session Event: TCP Connected 11/07/2024 02:20:52.590 7f90360d0e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:20:52.590 7f90360d0e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:20:52.590 7f90360d0e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:20:52.591 7f90360d0e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:20:52.591 7f90360d0e10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect] 11/07/2024 02:20:52.591 7f90360d0e10 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19 11/07/2024 02:20:52.591 7f90360d0e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 11/07/2024 02:20:52.591 7f90360d0e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/07/2024 02:20:52.591 7f90360d0e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/07/2024 02:20:52.592 7f90360e66d8 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 11/07/2024 02:20:52.593 7f90360e66d8 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 11/07/2024 02:20:52.596 7f90360e6588 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001017 11/07/2024 02:20:52.597 7f90360e6588 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 11/07/2024 02:20:52.598 7f903610be10 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 11/07/2024 02:20:52.598 7f903610be10 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 0 11/07/2024 02:20:52.598 7f903610be10 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 11/07/2024 02:20:52.598 7f903610be10 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/07/2024 02:20:52.598 7f903610be10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 11/07/2024 02:20:52.598 7f90360dc978 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 11/07/2024 02:20:52.600 7f90360dc978 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 11/07/2024 02:20:52.818 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:20:53.819 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:20:54.820 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:20:55.821 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:20:56.822 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:20:57.822 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:20:58.823 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:20:59.825 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:00.826 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:01.827 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:02.828 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:03.828 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:04.829 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:05.831 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:06.832 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:07.833 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:08.835 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:09.836 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:10.838 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:11.839 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:12.840 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:13.841 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:14.842 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:15.843 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:16.844 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:17.845 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:18.845 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:19.846 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:20.846 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:21.847 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:22.848 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:23.849 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:24.849 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:25.850 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:26.851 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:27.852 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:28.854 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:29.855 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:30.856 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:31.856 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:32.858 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:33.859 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:34.860 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:35.861 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:36.861 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:37.862 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:38.863 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:39.864 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:40.865 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:41.866 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:42.866 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:43.868 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:44.869 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:45.870 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:46.871 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:47.873 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:48.874 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:21:49.700 7f903ff8ff88 [contrail-device-manager] [ERROR]: network IPAM 'ipv6_link_local' already exists or other conflict: ['default-domain', 'default-project', '_internal_ipam_ipv6_link_local'] already exists with uuid: c7156525-7415-4bc0-8fb9-e096f9dc20b5 11/07/2024 02:21:49.852 7f903ff8ff88 [contrail-device-manager] [ERROR]: virtual network '_internal_vn_ipv6_link_local' already exists or other conflict: ['default-domain', 'default-project', '_internal_vn_ipv6_link_local'] already exists with uuid: 586c4e60-0a4a-42d0-9d86-197b2f9aa7c5 11/07/2024 02:21:49.875 7f903fbd0828 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:22:34.549 7f050a204648 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 11/07/2024 02:22:34.549 7f050a204648 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 11/07/2024 02:22:34.549 7f050a204648 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 11/07/2024 02:22:34.550 7f05018d46d8 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 11/07/2024 02:22:34.550 7f05018d46d8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 11/07/2024 02:22:34.550 7f05018d46d8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/07/2024 02:22:34.550 7f05018d46d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 11/07/2024 02:22:34.551 7f05018d46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:34.551 7f05018d46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:22:34.551 7f05018d46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:34.551 7f05018d46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:34.551 7f05018d46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 113 start_time = 1730946154550671 collector_name = collector_ip = collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 11/07/2024 02:22:34.551 7f05018d46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:34.585 7f050a204648 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.35:5673//'] 11/07/2024 02:22:34.589 7f050a204648 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 11/07/2024 02:22:34.591 7f0501825240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:34.591 7f0501825240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/07/2024 02:22:34.591 7f0501825240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:34.591 7f0501825240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:34.591 7f0501825240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/07/2024 02:22:34.592 7f0501825240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:34.599 7f050a204648 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 11/07/2024 02:22:34.600 7f0501825240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:34.600 7f0501825240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:22:34.600 7f0501825240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:34.606 7f050a204648 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 11/07/2024 02:22:34.606 7f050177c2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:34.607 7f050177c2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/07/2024 02:22:34.607 7f050177c2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:34.609 7f050a204648 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 11/07/2024 02:22:34.610 7f050177c2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:34.610 7f050177c2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:22:34.610 7f050177c2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:34.617 7f050a204648 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 11/07/2024 02:22:34.618 7f050a204648 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 11/07/2024 02:22:34.618 7f050a204648 [contrail-device-manager] [WARNING]: initializing cassandra cluster 11/07/2024 02:22:34.618 7f050a204648 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 11/07/2024 02:22:34.618 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 11/07/2024 02:22:34.618 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:22:34.619 7f05042e70f0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 11/07/2024 02:22:34.619 7f05042e70f0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 11/07/2024 02:22:34.619 7f050177ccc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:34.620 7f050177ccc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/07/2024 02:22:34.620 7f050177ccc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:34.621 7f050a204648 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 11/07/2024 02:22:34.625 7f0501757e10 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:22:34.625 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 11/07/2024 02:22:34.627 7f05042e70f0 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 11/07/2024 02:22:34.646 7f050a204648 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 11/07/2024 02:22:34.647 7f0500e1c8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:34.648 7f0500e1c8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 11/07/2024 02:22:34.648 7f0500e1c8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:34.653 7f050a204648 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 11/07/2024 02:22:34.654 7f0500e64828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:34.654 7f0500e64828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/07/2024 02:22:34.654 7f0500e64828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:34.658 7f050a204648 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 11/07/2024 02:22:34.666 7f050a204648 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 11/07/2024 02:22:34.666 7f0500e522e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:34.666 7f0500e522e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:22:34.667 7f0500e522e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:34.681 7f05017240f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:34.681 7f05017240f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:Zookeeper[] connection down >>, ] >> 11/07/2024 02:22:34.681 7f05017240f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:34.687 7f0500e56438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:34.687 7f0500e56438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:22:34.687 7f0500e56438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:34.687 7f050a204648 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 11/07/2024 02:22:34.687 7f050a204648 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 11/07/2024 02:22:34.687 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:22:34.688 7f050a204648 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1. 11/07/2024 02:22:34.690 7f0501757e10 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:22:34.696 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.'] 11/07/2024 02:22:34.702 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:22:34.706 7f0501757e10 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:22:34.714 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.'] 11/07/2024 02:22:35.201 7f050a204648 [contrail-device-manager] [DEBUG]: Fetching all containers 11/07/2024 02:22:35.471 7f050a204648 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 11/07/2024 02:22:35.666 7f050a204648 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..config_queue 11/07/2024 02:22:35.666 7f050a204648 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..tftp_queue 11/07/2024 02:22:35.715 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:22:35.722 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:22:35.726 7f0501757e10 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:22:35.736 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..tftp_queue'] 11/07/2024 02:22:36.167 7f050a204648 [contrail-device-manager] [WARNING]: Process 113 prepared to run in Full mode... 11/07/2024 02:22:36.167 7f050a204648 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 11/07/2024 02:22:36.167 7f050a204648 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 11/07/2024 02:22:36.170 7f050a204648 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page 11/07/2024 02:22:36.170 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 11/07/2024 02:22:36.171 7f050a204648 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page 11/07/2024 02:22:36.171 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 11/07/2024 02:22:36.172 7f050a204648 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 11/07/2024 02:22:36.172 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 11/07/2024 02:22:36.176 7f050a204648 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page 11/07/2024 02:22:36.176 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 11/07/2024 02:22:36.176 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 11/07/2024 02:22:36.176 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 11/07/2024 02:22:36.177 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 11/07/2024 02:22:36.177 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 11/07/2024 02:22:36.177 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 11/07/2024 02:22:36.177 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 11/07/2024 02:22:36.177 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 11/07/2024 02:22:36.177 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 11/07/2024 02:22:36.177 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 11/07/2024 02:22:36.177 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 11/07/2024 02:22:36.177 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 11/07/2024 02:22:36.179 7f050a204648 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 11/07/2024 02:22:36.179 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 11/07/2024 02:22:36.179 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 11/07/2024 02:22:36.179 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 11/07/2024 02:22:36.179 7f050a204648 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 11/07/2024 02:22:36.179 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 11/07/2024 02:22:36.180 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 11/07/2024 02:22:36.180 7f050a204648 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page 11/07/2024 02:22:36.180 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 11/07/2024 02:22:36.180 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 11/07/2024 02:22:36.180 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 11/07/2024 02:22:36.180 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 11/07/2024 02:22:36.180 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 11/07/2024 02:22:36.181 7f050a204648 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page 11/07/2024 02:22:36.181 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 11/07/2024 02:22:36.181 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 11/07/2024 02:22:36.181 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 11/07/2024 02:22:36.182 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 11/07/2024 02:22:36.182 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 11/07/2024 02:22:36.182 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 11/07/2024 02:22:36.182 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 11/07/2024 02:22:36.182 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 11/07/2024 02:22:36.182 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 11/07/2024 02:22:36.182 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 11/07/2024 02:22:36.183 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 11/07/2024 02:22:36.183 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 11/07/2024 02:22:36.183 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 11/07/2024 02:22:36.183 7f050a204648 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 11/07/2024 02:22:36.186 7f050a204648 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 11/07/2024 02:22:36.227 7f0500cd6048 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 11/07/2024 02:22:36.227 7f0500cd6048 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 11/07/2024 02:22:36.227 7f0500c79828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:36.227 7f0500c79828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[none to Idle on EvStart], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 11/07/2024 02:22:36.228 7f0500c79828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:36.244 7f0500ae8c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:36.245 7f0500ae8c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:22:36.245 7f0500ae8c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:22:36.739 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:22:36.747 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:22:36.757 7f0501757e10 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:22:36.776 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..tftp_queue', 'device_manager_ztp.ztp_queue'] 11/07/2024 02:22:37.779 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:22:38.551 7f050039ee10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:22:38.552 7f050039ee10 [contrail-device-manager] [INFO]: Session Event: TCP Connected 11/07/2024 02:22:38.552 7f050039ee10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:22:38.552 7f050039ee10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:22:38.552 7f050039ee10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:22:38.552 7f050039ee10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:22:38.552 7f050039ee10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect] 11/07/2024 02:22:38.552 7f050039ee10 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19 11/07/2024 02:22:38.553 7f050039ee10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 11/07/2024 02:22:38.553 7f050039ee10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/07/2024 02:22:38.553 7f050039ee10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/07/2024 02:22:38.553 7f05003a76d8 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 11/07/2024 02:22:38.554 7f05003a76d8 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 11/07/2024 02:22:38.556 7f05003a7588 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001017 11/07/2024 02:22:38.556 7f05003a7588 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 11/07/2024 02:22:38.557 7f0502481eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 11/07/2024 02:22:38.557 7f0502481eb8 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 0 11/07/2024 02:22:38.557 7f0502481eb8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 11/07/2024 02:22:38.557 7f0502481eb8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/07/2024 02:22:38.557 7f0502481eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 11/07/2024 02:22:38.557 7f05003a72e8 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 11/07/2024 02:22:38.559 7f05003a72e8 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 11/07/2024 02:22:38.781 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:22:39.782 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:22:40.783 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:22:41.784 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:22:42.785 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:22:43.786 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:22:44.787 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:22:45.788 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:22:46.789 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:22:47.790 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:22:48.791 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:22:49.792 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:22:50.793 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:22:51.794 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:22:52.795 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:22:53.796 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:22:54.797 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:22:55.798 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:22:56.799 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:22:57.801 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:22:58.802 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:22:59.802 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:00.804 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:01.805 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:02.806 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:03.808 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:04.808 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:05.810 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:06.811 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:07.812 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:08.815 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:09.816 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:10.817 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:11.818 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:12.821 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:13.822 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:14.823 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:15.825 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:16.826 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:17.826 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:18.827 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:19.828 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:20.829 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:21.830 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:22.831 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:23.832 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:24.833 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:25.834 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:26.836 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:27.837 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:28.838 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:29.839 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:30.839 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:31.841 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:32.842 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:33.843 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:34.844 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:35.845 7f0501757e10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:23:35.976 7f050a204648 [contrail-device-manager] [ERROR]: network IPAM 'ipv6_link_local' already exists or other conflict: ['default-domain', 'default-project', '_internal_ipam_ipv6_link_local'] already exists with uuid: c7156525-7415-4bc0-8fb9-e096f9dc20b5 11/07/2024 02:23:36.148 7f050a204648 [contrail-device-manager] [ERROR]: virtual network '_internal_vn_ipv6_link_local' already exists or other conflict: ['default-domain', 'default-project', '_internal_vn_ipv6_link_local'] already exists with uuid: 586c4e60-0a4a-42d0-9d86-197b2f9aa7c5 11/07/2024 02:24:20.578 7fa0fb33f248 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 11/07/2024 02:24:20.578 7fa0fb33f248 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 11/07/2024 02:24:20.578 7fa0fb33f248 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 11/07/2024 02:24:20.578 7fa0f2a0e6d8 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 11/07/2024 02:24:20.579 7fa0f2a0e6d8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 11/07/2024 02:24:20.579 7fa0f2a0e6d8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/07/2024 02:24:20.579 7fa0f2a0e6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 11/07/2024 02:24:20.579 7fa0f2a0e6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:20.580 7fa0f2a0e6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:24:20.580 7fa0f2a0e6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:20.580 7fa0f2a0e6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:20.580 7fa0f2a0e6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 113 start_time = 1730946260579409 collector_name = collector_ip = collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 11/07/2024 02:24:20.580 7fa0f2a0e6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:20.615 7fa0fb33f248 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.35:5673//'] 11/07/2024 02:24:20.620 7fa0fb33f248 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 11/07/2024 02:24:20.620 7fa0f2960240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:20.620 7fa0f2960240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/07/2024 02:24:20.620 7fa0f2960240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:20.621 7fa0f2960240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:20.621 7fa0f2960240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/07/2024 02:24:20.621 7fa0f2960240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:20.624 7fa0fb33f248 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 11/07/2024 02:24:20.624 7fa0f29600f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:20.624 7fa0f29600f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:24:20.624 7fa0f29600f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:20.630 7fa0fb33f248 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 11/07/2024 02:24:20.630 7fa0f28bb2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:20.631 7fa0f28bb2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/07/2024 02:24:20.631 7fa0f28bb2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:20.633 7fa0fb33f248 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 11/07/2024 02:24:20.634 7fa0f28bb2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:20.634 7fa0f28bb2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:24:20.634 7fa0f28bb2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:20.642 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 11/07/2024 02:24:20.642 7fa0fb33f248 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 11/07/2024 02:24:20.642 7fa0fb33f248 [contrail-device-manager] [WARNING]: initializing cassandra cluster 11/07/2024 02:24:20.642 7fa0fb33f248 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 11/07/2024 02:24:20.643 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 11/07/2024 02:24:20.643 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:24:20.643 7fa0f54230f0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 11/07/2024 02:24:20.644 7fa0f54230f0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 11/07/2024 02:24:20.644 7fa0f28bbcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:20.644 7fa0f28bbcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/07/2024 02:24:20.644 7fa0f28bbcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:20.645 7fa0fb33f248 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 11/07/2024 02:24:20.649 7fa0fb247780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:24:20.649 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 11/07/2024 02:24:20.650 7fa0f54230f0 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 11/07/2024 02:24:20.667 7fa0fb33f248 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 11/07/2024 02:24:20.668 7fa0f1f598d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:20.669 7fa0f1f598d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 11/07/2024 02:24:20.669 7fa0f1f598d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:20.674 7fa0fb33f248 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 11/07/2024 02:24:20.674 7fa0f1f9f828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:20.674 7fa0f1f9f828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/07/2024 02:24:20.674 7fa0f1f9f828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:20.679 7fa0fb33f248 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 11/07/2024 02:24:20.695 7fa0fb33f248 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 11/07/2024 02:24:20.696 7fa0f1f902e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:20.696 7fa0f1f902e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:24:20.696 7fa0f1f902e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:20.713 7fa0f1fc2240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:20.713 7fa0f1fc2240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:Zookeeper[] connection down >>, ] >> 11/07/2024 02:24:20.713 7fa0f1fc2240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:20.719 7fa0f1f8ed68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:20.719 7fa0f1f8ed68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:24:20.720 7fa0f1f8ed68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:20.720 7fa0fb33f248 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 11/07/2024 02:24:20.720 7fa0fb33f248 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 11/07/2024 02:24:20.720 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:24:20.721 7fa0fb33f248 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1. 11/07/2024 02:24:20.723 7fa0fb247780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:24:20.729 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.'] 11/07/2024 02:24:20.738 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:24:20.741 7fa0fb247780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:24:20.746 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.'] 11/07/2024 02:24:21.234 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Fetching all containers 11/07/2024 02:24:21.475 7fa0fb33f248 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 11/07/2024 02:24:21.749 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:21.819 7fa0fb33f248 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..config_queue 11/07/2024 02:24:21.820 7fa0fb33f248 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..tftp_queue 11/07/2024 02:24:22.320 7fa0fb33f248 [contrail-device-manager] [WARNING]: Process 113 prepared to run in Full mode... 11/07/2024 02:24:22.321 7fa0fb33f248 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 11/07/2024 02:24:22.321 7fa0fb33f248 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 11/07/2024 02:24:22.324 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page 11/07/2024 02:24:22.324 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 11/07/2024 02:24:22.325 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page 11/07/2024 02:24:22.325 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 11/07/2024 02:24:22.326 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 11/07/2024 02:24:22.326 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 11/07/2024 02:24:22.330 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page 11/07/2024 02:24:22.330 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 11/07/2024 02:24:22.330 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 11/07/2024 02:24:22.330 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 11/07/2024 02:24:22.330 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 11/07/2024 02:24:22.330 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 11/07/2024 02:24:22.330 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 11/07/2024 02:24:22.331 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 11/07/2024 02:24:22.331 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 11/07/2024 02:24:22.331 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 11/07/2024 02:24:22.331 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 11/07/2024 02:24:22.331 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 11/07/2024 02:24:22.331 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 11/07/2024 02:24:22.332 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 11/07/2024 02:24:22.333 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 11/07/2024 02:24:22.333 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 11/07/2024 02:24:22.333 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 11/07/2024 02:24:22.333 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 11/07/2024 02:24:22.333 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 11/07/2024 02:24:22.333 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 11/07/2024 02:24:22.334 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page 11/07/2024 02:24:22.334 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 11/07/2024 02:24:22.334 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 11/07/2024 02:24:22.334 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 11/07/2024 02:24:22.334 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 11/07/2024 02:24:22.334 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 11/07/2024 02:24:22.335 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page 11/07/2024 02:24:22.335 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 11/07/2024 02:24:22.335 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 11/07/2024 02:24:22.335 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 11/07/2024 02:24:22.335 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 11/07/2024 02:24:22.335 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 11/07/2024 02:24:22.335 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 11/07/2024 02:24:22.336 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 11/07/2024 02:24:22.336 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 11/07/2024 02:24:22.336 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 11/07/2024 02:24:22.336 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 11/07/2024 02:24:22.336 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 11/07/2024 02:24:22.336 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 11/07/2024 02:24:22.336 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 11/07/2024 02:24:22.336 7fa0fb33f248 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 11/07/2024 02:24:22.339 7fa0fb33f248 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 11/07/2024 02:24:22.379 7fa0f1cb2a20 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 11/07/2024 02:24:22.379 7fa0f1cb2a20 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 11/07/2024 02:24:22.379 7fa0f1f0b240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:22.380 7fa0f1f0b240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[none to Idle on EvStart], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 11/07/2024 02:24:22.380 7fa0f1f0b240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:22.394 7fa0f1daff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:22.394 7fa0f1daff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:24:22.394 7fa0f1daff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:24:22.750 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:22.756 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:24:22.759 7fa0fb247780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:24:22.777 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..tftp_queue', 'device_manager_ztp.ztp_queue'] 11/07/2024 02:24:23.779 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:24.579 7fa0f14e3a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:24:24.580 7fa0f14e3a20 [contrail-device-manager] [INFO]: Session Event: TCP Connected 11/07/2024 02:24:24.580 7fa0f14e3a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:24:24.580 7fa0f14e3a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:24:24.581 7fa0f14e3a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:24:24.581 7fa0f14e3a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:24:24.581 7fa0f14e3a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect] 11/07/2024 02:24:24.581 7fa0f14e3a20 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19 11/07/2024 02:24:24.581 7fa0f14e3a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 11/07/2024 02:24:24.581 7fa0f14e3a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/07/2024 02:24:24.581 7fa0f14e3a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/07/2024 02:24:24.581 7fa0f14e3ac8 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 11/07/2024 02:24:24.582 7fa0f14e3ac8 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 11/07/2024 02:24:24.585 7fa0f14e3d68 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001017 11/07/2024 02:24:24.586 7fa0f14e3d68 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 11/07/2024 02:24:24.586 7fa0f14e3390 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 11/07/2024 02:24:24.586 7fa0f14e3390 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 0 11/07/2024 02:24:24.586 7fa0f14e3390 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 11/07/2024 02:24:24.586 7fa0f14e3390 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/07/2024 02:24:24.587 7fa0f14e3390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 11/07/2024 02:24:24.587 7fa0f14e36d8 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 11/07/2024 02:24:24.588 7fa0f14e36d8 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 11/07/2024 02:24:24.780 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:25.781 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:26.782 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:27.783 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:28.784 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:29.785 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:30.787 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:31.788 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:32.789 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:33.789 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:34.791 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:35.792 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:36.793 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:37.794 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:38.795 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:39.796 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:40.797 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:41.799 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:42.800 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:43.801 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:44.802 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:45.803 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:46.804 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:47.805 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:48.806 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:49.807 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:50.808 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:51.809 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:52.810 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:53.811 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:54.812 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:55.813 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:56.813 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:57.814 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:58.816 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:24:59.817 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:25:00.818 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:25:01.818 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:25:02.819 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:25:03.820 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:25:04.822 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:25:05.823 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:25:06.825 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:25:07.827 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:25:08.828 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:25:09.829 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:25:10.830 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:25:11.831 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:25:12.833 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:25:13.835 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:25:14.836 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:25:15.837 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:25:16.838 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:25:17.839 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:25:18.841 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:25:19.842 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:25:20.808 7fa0fb33f248 [contrail-device-manager] [ERROR]: network IPAM 'ipv6_link_local' already exists or other conflict: ['default-domain', 'default-project', '_internal_ipam_ipv6_link_local'] already exists with uuid: c7156525-7415-4bc0-8fb9-e096f9dc20b5 11/07/2024 02:25:20.843 7fa0fb247780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:25:20.960 7fa0fb33f248 [contrail-device-manager] [ERROR]: virtual network '_internal_vn_ipv6_link_local' already exists or other conflict: ['default-domain', 'default-project', '_internal_vn_ipv6_link_local'] already exists with uuid: 586c4e60-0a4a-42d0-9d86-197b2f9aa7c5 11/07/2024 02:26:06.595 7f31971b4ac8 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 11/07/2024 02:26:06.595 7f31971b4ac8 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 11/07/2024 02:26:06.595 7f31971b4ac8 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 11/07/2024 02:26:06.595 7f318e8226d8 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 11/07/2024 02:26:06.596 7f318e8226d8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 11/07/2024 02:26:06.596 7f318e8226d8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/07/2024 02:26:06.596 7f318e8226d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 11/07/2024 02:26:06.596 7f318e8226d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:06.596 7f318e8226d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:26:06.596 7f318e8226d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:06.596 7f318e8226d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:06.597 7f318e8226d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 113 start_time = 1730946366596396 collector_name = collector_ip = collector_list = [ 10.0.0.35:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 11/07/2024 02:26:06.597 7f318e8226d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:06.630 7f31971b4ac8 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.35:5673//'] 11/07/2024 02:26:06.635 7f31971b4ac8 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 11/07/2024 02:26:06.635 7f318e772240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:06.636 7f318e772240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/07/2024 02:26:06.636 7f318e772240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:06.636 7f318e772240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:06.636 7f318e772240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/07/2024 02:26:06.636 7f318e772240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:06.640 7f31971b4ac8 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 11/07/2024 02:26:06.640 7f318e7720f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:06.640 7f318e7720f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:26:06.640 7f318e7720f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:06.650 7f31971b4ac8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 11/07/2024 02:26:06.651 7f318e6ca2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:06.651 7f318e6ca2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/07/2024 02:26:06.651 7f318e6ca2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:06.654 7f31971b4ac8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 11/07/2024 02:26:06.654 7f318e6ca2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:06.654 7f318e6ca2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:26:06.654 7f318e6ca2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:06.661 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 11/07/2024 02:26:06.661 7f31971b4ac8 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 11/07/2024 02:26:06.661 7f31971b4ac8 [contrail-device-manager] [WARNING]: initializing cassandra cluster 11/07/2024 02:26:06.662 7f31971b4ac8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 11/07/2024 02:26:06.662 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 11/07/2024 02:26:06.662 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:26:06.662 7f31912360f0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 11/07/2024 02:26:06.663 7f31912360f0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 11/07/2024 02:26:06.663 7f318e6cacc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:06.663 7f318e6cacc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/07/2024 02:26:06.663 7f318e6cacc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:06.664 7f31971b4ac8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 11/07/2024 02:26:06.668 7f31912360f0 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 11/07/2024 02:26:06.668 7f3196df8780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:26:06.668 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 11/07/2024 02:26:06.688 7f31971b4ac8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 11/07/2024 02:26:06.689 7f318e56e780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:06.689 7f318e56e780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 11/07/2024 02:26:06.689 7f318e56e780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:06.694 7f31971b4ac8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 11/07/2024 02:26:06.694 7f318e571c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:06.694 7f318e571c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/07/2024 02:26:06.694 7f318e571c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:06.698 7f31971b4ac8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 11/07/2024 02:26:06.709 7f31971b4ac8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 11/07/2024 02:26:06.709 7f318e57f6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:06.709 7f318e57f6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:26:06.709 7f318e57f6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:06.724 7f318e68a048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:06.724 7f318e68a048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:Zookeeper[] connection down >>, ] >> 11/07/2024 02:26:06.724 7f318e68a048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:06.730 7f318e6ca4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:06.730 7f318e6ca4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:26:06.731 7f318e6ca4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:06.731 7f31971b4ac8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 11/07/2024 02:26:06.731 7f31971b4ac8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 11/07/2024 02:26:06.731 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:26:06.732 7f31971b4ac8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1. 11/07/2024 02:26:06.735 7f3196df8780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:26:06.741 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.'] 11/07/2024 02:26:06.747 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:26:06.750 7f3196df8780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:26:06.754 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.'] 11/07/2024 02:26:07.244 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Fetching all containers 11/07/2024 02:26:07.493 7f31971b4ac8 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 11/07/2024 02:26:07.756 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:08.208 7f31971b4ac8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..config_queue 11/07/2024 02:26:08.209 7f31971b4ac8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..tftp_queue 11/07/2024 02:26:08.709 7f31971b4ac8 [contrail-device-manager] [WARNING]: Process 113 prepared to run in Full mode... 11/07/2024 02:26:08.709 7f31971b4ac8 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 11/07/2024 02:26:08.710 7f31971b4ac8 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 11/07/2024 02:26:08.713 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page 11/07/2024 02:26:08.713 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 11/07/2024 02:26:08.713 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page 11/07/2024 02:26:08.714 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 11/07/2024 02:26:08.714 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 11/07/2024 02:26:08.715 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 11/07/2024 02:26:08.718 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page 11/07/2024 02:26:08.718 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 11/07/2024 02:26:08.718 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 11/07/2024 02:26:08.719 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 11/07/2024 02:26:08.719 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 11/07/2024 02:26:08.719 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 11/07/2024 02:26:08.719 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 11/07/2024 02:26:08.719 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 11/07/2024 02:26:08.719 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 11/07/2024 02:26:08.719 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 11/07/2024 02:26:08.719 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 11/07/2024 02:26:08.719 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 11/07/2024 02:26:08.719 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 11/07/2024 02:26:08.721 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 11/07/2024 02:26:08.721 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 11/07/2024 02:26:08.721 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 11/07/2024 02:26:08.721 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 11/07/2024 02:26:08.721 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 11/07/2024 02:26:08.721 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 11/07/2024 02:26:08.722 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 11/07/2024 02:26:08.722 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page 11/07/2024 02:26:08.722 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 11/07/2024 02:26:08.722 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 11/07/2024 02:26:08.722 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 11/07/2024 02:26:08.722 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 11/07/2024 02:26:08.722 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 11/07/2024 02:26:08.723 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page 11/07/2024 02:26:08.723 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 11/07/2024 02:26:08.723 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 11/07/2024 02:26:08.723 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 11/07/2024 02:26:08.723 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 11/07/2024 02:26:08.724 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 11/07/2024 02:26:08.724 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 11/07/2024 02:26:08.724 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 11/07/2024 02:26:08.724 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 11/07/2024 02:26:08.724 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 11/07/2024 02:26:08.724 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 11/07/2024 02:26:08.724 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 11/07/2024 02:26:08.724 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 11/07/2024 02:26:08.724 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 11/07/2024 02:26:08.724 7f31971b4ac8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 11/07/2024 02:26:08.727 7f31971b4ac8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 11/07/2024 02:26:08.766 7f318e2c5f60 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 11/07/2024 02:26:08.766 7f318e2c5f60 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 11/07/2024 02:26:08.766 7f318e3094e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:08.767 7f318e3094e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[none to Idle on EvStart], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 11/07/2024 02:26:08.767 7f318e3094e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:08.767 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:08.782 7f318e2e8390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:08.782 7f318e2e8390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/07/2024 02:26:08.783 7f318e2e8390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/07/2024 02:26:08.783 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 11/07/2024 02:26:08.786 7f3196df8780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 11/07/2024 02:26:08.797 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2207-1.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2207-1..tftp_queue', 'device_manager_ztp.ztp_queue'] 11/07/2024 02:26:09.799 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:10.597 7f318daeeeb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/07/2024 02:26:10.598 7f318daeeeb8 [contrail-device-manager] [INFO]: Session Event: TCP Connected 11/07/2024 02:26:10.598 7f318daeeeb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/07/2024 02:26:10.598 7f318daeeeb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:26:10.598 7f318daeeeb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2207-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.35:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.35:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.35:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.35:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.35:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/07/2024 02:26:10.598 7f318daeeeb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/07/2024 02:26:10.598 7f318daeeeb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect] 11/07/2024 02:26:10.598 7f318daeeeb8 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19 11/07/2024 02:26:10.599 7f318daeeeb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 11/07/2024 02:26:10.599 7f318daeeeb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/07/2024 02:26:10.599 7f318daeeeb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/07/2024 02:26:10.599 7f318e516d68 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 11/07/2024 02:26:10.600 7f318e516d68 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 11/07/2024 02:26:10.602 7f318dba2780 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001017 11/07/2024 02:26:10.603 7f318dba2780 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 11/07/2024 02:26:10.603 7f318e57c438 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 11/07/2024 02:26:10.603 7f318e57c438 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 0 11/07/2024 02:26:10.604 7f318e57c438 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 11/07/2024 02:26:10.604 7f318e57c438 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/07/2024 02:26:10.604 7f318e57c438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 11/07/2024 02:26:10.604 7f318daeeeb8 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 11/07/2024 02:26:10.605 7f318daeeeb8 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 11/07/2024 02:26:10.800 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:11.802 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:12.803 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:13.804 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:14.805 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:15.806 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:16.807 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:17.809 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:18.809 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:19.811 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:20.812 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:21.812 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:22.813 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:23.814 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:24.815 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:25.817 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:26.818 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:27.819 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:28.821 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:29.822 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:30.823 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:31.824 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:32.826 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:33.827 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:34.827 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:35.828 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:36.829 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:37.776 7f318daf4978 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 11/07/2024 02:26:37.830 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:38.832 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:39.833 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:40.834 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:41.835 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 11/07/2024 02:26:42.835 7f3196df8780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: socket.timeout occurred while draining events, no events to process.