02/10/2025 01:55:51.780 7fa6a59d10c8 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 02/10/2025 01:55:51.780 7fa6a59d10c8 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 02/10/2025 01:55:51.780 7fa6a59d10c8 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 02/10/2025 01:55:51.780 7fa69d0d6b70 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 02/10/2025 01:55:51.781 7fa69d0d6b70 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 02/10/2025 01:55:51.781 7fa69d0d6b70 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/10/2025 01:55:51.781 7fa69d0d6b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 02/10/2025 01:55:51.781 7fa69d0d6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:55:51.782 7fa69d0d6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2705-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/10/2025 01:55:51.782 7fa69d0d6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:55:51.782 7fa69d0d6b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:55:51.782 7fa69d0d6b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2705-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 139 start_time = 1739152551781356 collector_name = collector_ip = collector_list = [ 10.0.0.17:8086, 10.0.0.12:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 02/10/2025 01:55:51.782 7fa69d0d6b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:55:51.897 7fa6a59d10c8 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.17:5673//', 'pyamqp://guest:guest@10.0.0.12:5673//', 'pyamqp://guest:guest@10.0.0.30:5673//'] 02/10/2025 01:55:51.949 7fa6a59d10c8 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 02/10/2025 01:55:51.950 7fa69d083198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:55:51.950 7fa69d083198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2705-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, 10.0.0.12:5673, 10.0.0.30:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 02/10/2025 01:55:51.950 7fa69d083198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:55:51.950 7fa69d083198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:55:51.950 7fa69d083198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2705-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, 10.0.0.12:5673, 10.0.0.30:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 02/10/2025 01:55:51.950 7fa69d083198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:55:51.956 7fa6a59d10c8 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 02/10/2025 01:55:51.956 7fa69cf41b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:55:51.956 7fa69cf41b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2705-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, 10.0.0.12:5673, 10.0.0.30:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/10/2025 01:55:51.956 7fa69cf41b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:55:52.093 7fa6a59d10c8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 02/10/2025 01:55:52.094 7fa69d083438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:55:52.094 7fa69d083438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2705-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, 10.0.0.12:5673, 10.0.0.30:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 02/10/2025 01:55:52.094 7fa69d083438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:55:52.097 7fa6a59d10c8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 02/10/2025 01:55:52.097 7fa69d083198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:55:52.098 7fa69d083198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2705-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, 10.0.0.12:5673, 10.0.0.30:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/10/2025 01:55:52.098 7fa69d083198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:55:52.238 7fa6a59d10c8 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 02/10/2025 01:55:52.239 7fa6a59d10c8 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 02/10/2025 01:55:52.239 7fa6a59d10c8 [contrail-device-manager] [WARNING]: initializing cassandra cluster 02/10/2025 01:55:52.239 7fa6a59d10c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 02/10/2025 01:55:52.239 7fa6a2947780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 02/10/2025 01:55:52.239 7fa6a2947780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 02/10/2025 01:55:52.240 7fa69d07de10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 02/10/2025 01:55:52.240 7fa69d07de10 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 02/10/2025 01:55:52.240 7fa69cea2438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:55:52.240 7fa69cea2438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2705-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, 10.0.0.12:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, 10.0.0.12:9041, 10.0.0.30:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 02/10/2025 01:55:52.240 7fa69cea2438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:55:52.242 7fa6a59d10c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 02/10/2025 01:55:52.243 7fa69d07de10 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 02/10/2025 01:55:52.244 7fa6a2947780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 02/10/2025 01:55:52.244 7fa6a2947780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 02/10/2025 01:55:52.269 7fa6a59d10c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 02/10/2025 01:55:52.272 7fa69ce35198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:55:52.272 7fa69ce35198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2705-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, 10.0.0.12:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, 10.0.0.12:9041, 10.0.0.30:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, 10.0.0.12:2181, 10.0.0.30:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 02/10/2025 01:55:52.272 7fa69ce35198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:55:55.782 7fa69cf41780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/10/2025 01:55:55.783 7fa69cf41780 [contrail-device-manager] [INFO]: Session Event: TCP Connected 02/10/2025 01:55:55.783 7fa69cf41780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/10/2025 01:55:55.783 7fa69cf41780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/10/2025 01:55:55.783 7fa69cf41780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2705-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, 10.0.0.12:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, 10.0.0.12:9041, 10.0.0.30:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, 10.0.0.12:2181, 10.0.0.30:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 02/10/2025 01:55:55.783 7fa69cf41780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/10/2025 01:55:55.783 7fa69cf41780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/10/2025 01:55:55.783 7fa69cf41780 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19 02/10/2025 01:55:55.784 7fa69cf41780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/10/2025 01:55:55.784 7fa69cf41780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/10/2025 01:55:55.784 7fa69cf41780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/10/2025 01:55:55.784 7fa6a087fb70 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/10/2025 01:55:55.785 7fa6a087fb70 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/10/2025 01:55:55.792 7fa69d0d68d0 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001017 02/10/2025 01:55:55.792 7fa69d0d68d0 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/10/2025 01:55:55.792 7fa69ce90240 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 02/10/2025 01:55:55.792 7fa69ce90240 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 0 02/10/2025 01:55:55.793 7fa69ce90240 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/10/2025 01:55:55.793 7fa69ce90240 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/10/2025 01:55:55.793 7fa69ce90240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 02/10/2025 01:55:55.793 7fa6a087fb70 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/10/2025 01:55:55.794 7fa6a087fb70 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/10/2025 01:56:57.040 7fbc3d7e1708 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 02/10/2025 01:56:57.046 7fbc3d7e1708 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 02/10/2025 01:56:57.046 7fbc3d7e1708 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 02/10/2025 01:56:57.047 7fbc34ef2b70 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 02/10/2025 01:56:57.047 7fbc34ef2b70 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 02/10/2025 01:56:57.048 7fbc34ef2b70 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/10/2025 01:56:57.048 7fbc34ef2b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 02/10/2025 01:56:57.048 7fbc34ef2b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:57.048 7fbc34ef2b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2705-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/10/2025 01:56:57.048 7fbc34ef2b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:57.048 7fbc34ef2b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:57.048 7fbc34ef2b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2705-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1739152617048025 collector_name = collector_ip = collector_list = [ 10.0.0.17:8086, 10.0.0.30:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 02/10/2025 01:56:57.048 7fbc34ef2b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:57.109 7fbc3d7e1708 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.17:5673//', 'pyamqp://guest:guest@10.0.0.12:5673//', 'pyamqp://guest:guest@10.0.0.30:5673//'] 02/10/2025 01:56:57.121 7fbc3d7e1708 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 02/10/2025 01:56:57.124 7fbc34d4e048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:57.124 7fbc34d4e048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2705-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, 10.0.0.12:5673, 10.0.0.30:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 02/10/2025 01:56:57.124 7fbc34d4e048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:57.124 7fbc34d4e048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:57.124 7fbc34d4e048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2705-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, 10.0.0.12:5673, 10.0.0.30:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 02/10/2025 01:56:57.124 7fbc34d4e048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:57.131 7fbc3d7e1708 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 02/10/2025 01:56:57.131 7fbc34d4e198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:57.131 7fbc34d4e198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2705-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, 10.0.0.12:5673, 10.0.0.30:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/10/2025 01:56:57.131 7fbc34d4e198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:57.173 7fbc3d7e1708 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 02/10/2025 01:56:57.174 7fbc34ce18d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:57.174 7fbc34ce18d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2705-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, 10.0.0.12:5673, 10.0.0.30:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 02/10/2025 01:56:57.174 7fbc34ce18d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:57.179 7fbc3d7e1708 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 02/10/2025 01:56:57.179 7fbc436dc978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:57.179 7fbc436dc978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2705-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, 10.0.0.12:5673, 10.0.0.30:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/10/2025 01:56:57.179 7fbc436dc978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:57.193 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 02/10/2025 01:56:57.193 7fbc3d7e1708 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 02/10/2025 01:56:57.194 7fbc3d7e1708 [contrail-device-manager] [WARNING]: initializing cassandra cluster 02/10/2025 01:56:57.194 7fbc3d7e1708 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 02/10/2025 01:56:57.194 7fbc3a762780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 02/10/2025 01:56:57.194 7fbc3a762780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 02/10/2025 01:56:57.195 7fbc34e19780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 02/10/2025 01:56:57.195 7fbc34e19780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 02/10/2025 01:56:57.195 7fbc34d01198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:57.195 7fbc34d01198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2705-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, 10.0.0.12:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, 10.0.0.12:9041, 10.0.0.30:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 02/10/2025 01:56:57.195 7fbc34d01198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:57.197 7fbc3d7e1708 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 02/10/2025 01:56:57.199 7fbc34e19780 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 02/10/2025 01:56:57.210 7fbc3a762780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 02/10/2025 01:56:57.210 7fbc3a762780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 02/10/2025 01:56:57.247 7fbc3d7e1708 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 02/10/2025 01:56:57.249 7fbc34bcda20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:57.250 7fbc34bcda20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2705-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, 10.0.0.12:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, 10.0.0.12:9041, 10.0.0.30:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, 10.0.0.12:2181, 10.0.0.30:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 02/10/2025 01:56:57.250 7fbc34bcda20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:57.261 7fbc3d7e1708 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 02/10/2025 01:56:57.262 7fbc34bcda20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:57.262 7fbc34bcda20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2705-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, 10.0.0.12:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, 10.0.0.12:9041, 10.0.0.30:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, 10.0.0.12:2181, 10.0.0.30:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 02/10/2025 01:56:57.262 7fbc34bcda20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:57.278 7fbc3d7e1708 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 02/10/2025 01:56:57.315 7fbc3d7e1708 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 02/10/2025 01:56:57.316 7fbc34bf9198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:57.316 7fbc34bf9198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2705-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, 10.0.0.12:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, 10.0.0.12:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, 10.0.0.12:2181, 10.0.0.30:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/10/2025 01:56:57.316 7fbc34bf9198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:57.416 7fbc3437c4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:57.417 7fbc3437c4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2705-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, 10.0.0.12:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, 10.0.0.12:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, 10.0.0.12:2181, 10.0.0.30:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:Zookeeper[] connection down >>, ] >> 02/10/2025 01:56:57.417 7fbc3437c4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:57.432 7fbc3437e390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:57.432 7fbc3437e390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2705-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, 10.0.0.12:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, 10.0.0.12:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, 10.0.0.12:2181, 10.0.0.30:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/10/2025 01:56:57.432 7fbc3437e390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:57.433 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 02/10/2025 01:56:57.433 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 02/10/2025 01:56:57.433 7fbc3a762780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 02/10/2025 01:56:57.433 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2705-3. 02/10/2025 01:56:57.436 7fbc3a762780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 02/10/2025 01:56:57.482 7fbc3a762780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2705-3.'] 02/10/2025 01:56:57.503 7fbc3a762780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 02/10/2025 01:56:57.511 7fbc3a762780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 02/10/2025 01:56:57.557 7fbc3a762780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2705-3.'] 02/10/2025 01:56:58.028 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Fetching all containers 02/10/2025 01:56:58.318 7fbc3d7e1708 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 02/10/2025 01:56:58.656 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2705-3..config_queue 02/10/2025 01:56:58.657 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2705-3..tftp_queue 02/10/2025 01:56:59.157 7fbc3d7e1708 [contrail-device-manager] [WARNING]: Process 140 prepared to run in Full mode... 02/10/2025 01:56:59.157 7fbc3d7e1708 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 02/10/2025 01:56:59.158 7fbc3d7e1708 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 02/10/2025 01:56:59.160 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page 02/10/2025 01:56:59.160 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 02/10/2025 01:56:59.161 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page 02/10/2025 01:56:59.161 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 02/10/2025 01:56:59.162 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 02/10/2025 01:56:59.162 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 02/10/2025 01:56:59.166 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page 02/10/2025 01:56:59.166 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 02/10/2025 01:56:59.166 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 02/10/2025 01:56:59.166 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 02/10/2025 01:56:59.166 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 02/10/2025 01:56:59.166 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 02/10/2025 01:56:59.166 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 02/10/2025 01:56:59.166 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 02/10/2025 01:56:59.166 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 02/10/2025 01:56:59.166 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 02/10/2025 01:56:59.166 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 02/10/2025 01:56:59.166 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 02/10/2025 01:56:59.167 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 02/10/2025 01:56:59.168 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 02/10/2025 01:56:59.168 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 02/10/2025 01:56:59.168 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 02/10/2025 01:56:59.168 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 02/10/2025 01:56:59.168 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 02/10/2025 01:56:59.168 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 02/10/2025 01:56:59.168 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 02/10/2025 01:56:59.169 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page 02/10/2025 01:56:59.169 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 02/10/2025 01:56:59.169 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 02/10/2025 01:56:59.169 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 02/10/2025 01:56:59.169 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 02/10/2025 01:56:59.169 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 02/10/2025 01:56:59.170 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page 02/10/2025 01:56:59.170 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 02/10/2025 01:56:59.170 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 02/10/2025 01:56:59.170 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 02/10/2025 01:56:59.170 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 02/10/2025 01:56:59.170 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 02/10/2025 01:56:59.170 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 02/10/2025 01:56:59.170 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 02/10/2025 01:56:59.171 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 02/10/2025 01:56:59.171 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 02/10/2025 01:56:59.171 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 02/10/2025 01:56:59.171 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 02/10/2025 01:56:59.171 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 02/10/2025 01:56:59.172 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 02/10/2025 01:56:59.172 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 02/10/2025 01:56:59.180 7fbc3d7e1708 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 02/10/2025 01:56:59.295 7fbc34380cc0 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 02/10/2025 01:56:59.296 7fbc34380cc0 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 02/10/2025 01:56:59.297 7fbc2fa120f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:59.300 7fbc2fa120f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2705-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, 10.0.0.12:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, 10.0.0.12:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, 10.0.0.12:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.17,10.0.0.12,10.0.0.30:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[none to Idle on EvStart], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 02/10/2025 01:56:59.300 7fbc2fa120f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:59.483 7fbc2fa52cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:59.483 7fbc2fa52cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2705-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, 10.0.0.12:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, 10.0.0.12:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, 10.0.0.12:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.17,10.0.0.12,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/10/2025 01:56:59.483 7fbc2fa52cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/10/2025 01:56:59.584 7fbc3a762780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 02/10/2025 01:56:59.593 7fbc3a762780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 02/10/2025 01:56:59.925 7fbc3a762780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2705-3.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2705-3..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2705-3..tftp_queue', 'device_manager_ztp.ztp_queue'] 02/10/2025 01:57:01.048 7fbc2f93e588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/10/2025 01:57:01.050 7fbc2f93e588 [contrail-device-manager] [INFO]: Session Event: TCP Connected 02/10/2025 01:57:01.050 7fbc2f93e588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/10/2025 01:57:01.050 7fbc2f93e588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/10/2025 01:57:01.050 7fbc2f93e588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2705-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, 10.0.0.12:5673, 10.0.0.30:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, 10.0.0.12:9041, 10.0.0.30:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, 10.0.0.12:2181, 10.0.0.30:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.17,10.0.0.12,10.0.0.30:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/10/2025 01:57:01.050 7fbc2f93e588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/10/2025 01:57:01.050 7fbc2f93e588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/10/2025 01:57:01.050 7fbc2f93e588 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19 02/10/2025 01:57:01.051 7fbc2f93e588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/10/2025 01:57:01.051 7fbc2f93e588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/10/2025 01:57:01.051 7fbc2f93e588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/10/2025 01:57:01.051 7fbc3b701978 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/10/2025 01:57:01.052 7fbc3b701978 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/10/2025 01:57:01.056 7fbc2f942b70 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001017 02/10/2025 01:57:01.057 7fbc2f942b70 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/10/2025 01:57:01.057 7fbc2f935f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 02/10/2025 01:57:01.057 7fbc2f935f60 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 0 02/10/2025 01:57:01.057 7fbc2f935f60 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/10/2025 01:57:01.057 7fbc2f935f60 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/10/2025 01:57:01.057 7fbc2f935f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 02/10/2025 01:57:01.058 7fbc2f93e588 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/10/2025 01:57:01.059 7fbc2f93e588 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/10/2025 01:57:32.976 7fbc3d7e1708 [contrail-device-manager] [ERROR]: network IPAM 'ipv6_link_local' already exists or other conflict: ['default-domain', 'default-project', '_internal_ipam_ipv6_link_local'] already exists with uuid: 17731666-a2b0-4ded-ab05-94123e54d02c 02/10/2025 01:57:33.068 7fbc3d7e1708 [contrail-device-manager] [ERROR]: virtual network '_internal_vn_ipv6_link_local' already exists or other conflict: ['default-domain', 'default-project', '_internal_vn_ipv6_link_local'] already exists with uuid: 9816ff74-fff9-44f1-b27e-18cc06d73ad2 02/10/2025 01:57:33.386 7fbc3d7e1708 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 02/10/2025 01:57:33.386 7fbc3d7e1708 [contrail-device-manager] [WARNING]: initializing cassandra cluster 02/10/2025 01:57:33.386 7fbc3d7e1708 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 02/10/2025 01:57:33.388 7fbc3d7e1708 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 02/10/2025 01:57:33.458 7fbc3d7e1708 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 02/10/2025 01:57:33.466 7fbc3d7e1708 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 02/10/2025 01:57:33.473 7fbc3d7e1708 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 02/10/2025 01:57:33.613 7fbc3d7e1708 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 02/10/2025 01:57:33.691 7fbc3d7e1708 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.17:5673//', 'pyamqp://guest:guest@10.0.0.12:5673//', 'pyamqp://guest:guest@10.0.0.30:5673//'] 02/10/2025 01:57:33.692 7fbc3d7e1708 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 02/10/2025 01:57:33.692 7fbc34e19630 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/10/2025 01:57:33.693 7fbc34e19630 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/10/2025 01:57:33.697 7fbc3d7e1708 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 02/10/2025 01:57:33.731 7fbc3d7e1708 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 02/10/2025 01:57:33.743 7fbc3d7e1708 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 02/10/2025 01:57:33.746 7fbc34e8d4e0 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 02/10/2025 01:57:33.759 7fbc34e8d4e0 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED